erpc-io / erpc Goto Github PK

View Code? Open in Web Editor NEW

832.0 832.0 135.0 6.78 MB

Efficient RPCs for datacenter networks

Home Page: https://erpc.io/

License: Other

Python 2.92% C++ 74.38% CMake 1.14% Shell 2.39% C 19.08% Makefile 0.10%

erpc's People

Contributors

Stargazers

Watchers

Forkers

tchen0123 aljer bistack skyformat99 shawnshanksgui tigerjibo gjerecze theseusyang qiuhaonan bolinov ariesdevil kramelec ajiang38740 jakubradtke daq-db shunpeizhang imcg alexyu0 jnsquire chlin501 dhilip89 vsbenas linecode zzqq2199 parveshgarg ggerard clarephang anku94 shanling2004 lionxu denny-zhao lpstudy gakalaba yilongli lemonchar angelajiang leonlee666 dorealab jungan yangzhou1997 sirrainman beyer-yan hequan xiyangge jvikstrom yongming spongeann xuefengchang l-nic steamgjk theojepsen mrgaoxx garywangcn skpupil yanjun97 gbxtenfour depaulmillz beyonddream-productions zhao6300 nilyibo runmin oxooo pochtmeister fattaholmanan saendigphilip pickinguppieces jim90247 howard0su kinderriven gancc123 songhexiang srikanth2212 shenweihai1 leebiq xiaozuonanjing vbirds kyleshu zeroone7568 tidesq jacnel psistakis johnny5188 jitaogithub caterby global-localhost global19 global19-atlassian-net nifsem cnutshell leomikezee pdsl-dpu-kv iciclef gaishun soujanyaponnapalli-msr-22 szpgit staywizu huyk18 shuoshuc 5gapp zhulinqingfeng2018

erpc's Issues

dpdk support

Nice project! It's what I am looking for, is dpdk support production ready?

A few compile warnings and errors in applications

Hi Anuj,

Thank you for open-sourcing eRPC and keeping it well-maintained. I can build apps like "small_rpc_tput" if I take out Werror, but I encounter some errors when building other apps. I raise this issue to ask for some hints if possible :)

First of all, my current setting (CloudLab Apt Utah):
NIC: Connect-X3 pro (IB)
driver: MLXN_OFED 4.2-1.0.0.0
OS: ubuntu-14.04
gcc: 8.1.0
cmake: 3.12.0
gflags: 2.2.1
glogs: 0.3.5
hugepages and shared memory are configured
and I'm using InfiniBand
I configure cmake using cmake . -DPERF=ON -DTRANSPORT=infiniband
I'm using commit (latest one at master)

Now the issues:
<1> Compile Warnings: basically 2 types.
-warning (1):

In file included from /users/yiwenzhg/eRPC/src/rpc_impl/rpc_queues.cc:1:
/users/yiwenzhg/eRPC/src/rpc.h: In instantiation of ‘void erpc::Rpc<T>::reset_dpath_stats() [with TTr = erpc::IBTransport]’:
/users/yiwenzhg/eRPC/src/rpc_impl/rpc_queues.cc:80:1:   required from here
/users/yiwenzhg/eRPC/src/rpc.h:366:36: warning: ‘void* memset(void*, int, size_t)’ clearing an object of non-trivial type ‘struct erpc::Rpc<erpc::IBTransport>::<unnamed>’; use assignment or value-initialization instead [-Wclass-memaccess]
   void reset_dpath_stats() { memset(&dpath_stats, 0, sizeof(dpath_stats)); }
                              ~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/users/yiwenzhg/eRPC/src/rpc.h:1011:10: note: ‘struct erpc::Rpc<erpc::IBTransport>::<unnamed>’ declared here
   struct {

-warning (2):

cc1plus: warning: unrecognized command line option ‘-Wno-keyword-macro’
cc1plus: warning: unrecognized command line option ‘-Wno-nested-anon-types’

<2> build error in app "latency":

/users/yiwenzhg/eRPC/apps/latency/latency.cc:106:26: error: variable or field ‘app_cont_func’ declared void
 void app_cont_func(erpc::RespHandle *, void *, size_t);
                          ^~~~~~~~~~
/users/yiwenzhg/eRPC/apps/latency/latency.cc:106:26: error: ‘RespHandle’ is not a member of ‘erpc’
/users/yiwenzhg/eRPC/apps/latency/latency.cc:106:26: note: suggested alternative: ‘ReqHandle’
 void app_cont_func(erpc::RespHandle *, void *, size_t);

<3> build error in app "masstree-analytics":

cc1plus: fatal error: /users/yiwenzhg/masstree-beta/config.h: No such file or directory

which looks for config.h under anujkaliaiitd/masstree-beta.

Thanks!

Best regards,
Yiwen

Low Performance using IB with CX-3

Hi Anuj,

I tried eRPC small_rpc_tput but got low performance in per-thread throughput (~0.4 or ~0.5 Mrps vs. ~4 Mrps mentioned in the eRPC paper) and I'm wondering where I did wrong. I'll post my setting and app config below.

First, my experiment setting (CloudLab Apt Utah, C6220 node):
NIC: Connect-X3 pro (InfiniBand)
driver: MLXN_OFED 4.2-1.0.0.0
OS: ubuntu-14.04
gcc: 8.1.0
cmake: 3.12.0
gflags: 2.2.1
glogs: 0.3.5
hugepages and shared memory are configured
I configure cmake using cmake . -DPERF=ON -DTRANSPORT=infiniband
I'm using the lasted commit of the master branch.

Now the small_rpc_tput configuration:
--test_ms 20000
--sm_verbose 0
--batch_size 5
--concurrency 60
--msg_size 32
--num_processes 2
--num_threads 4
--numa_0_ports 0
--numa_1_ports 1,3

And a snapshot of the printout:
Process 0, thread 0: 0.488 Mrps, re_tx = 0, still_in_wheel = 0. RX: 487K resps, 487K reqs. Resps/batch: min 8K, max 8K. Latency: N/A. Rate = [0.12, 0.12, 0.24, 0.28 Gbps].
Process 0, thread 1: 0.485 Mrps, re_tx = 0, still_in_wheel = 0. RX: 484K resps, 487K reqs. Resps/batch: min 8K, max 8K. Latency: N/A. Rate = [0.12, 0.12, 0.16, 0.24 Gbps].
Process 0, thread 3: 0.486 Mrps, re_tx = 0, still_in_wheel = 0. RX: 485K resps, 485K reqs. Resps/batch: min 8K, max 8K. Latency: N/A. Rate = [0.12, 0.12, 0.16, 0.99 Gbps].
Process 0, thread 2: 0.485 Mrps, re_tx = 0, still_in_wheel = 0. RX: 485K resps, 485K reqs. Resps/batch: min 8K, max 8K. Latency: N/A. Rate = [0.12, 0.12, 0.21, 0.46 Gbps].

I tried to debug this issue, and here are my current findings:
(1) The Timely Rate number seems to be small. What does this rate mean and is Timely working properly?
(2) Increasing batch_size and adjusting concurrency does not affect the performance.
(3) Increasing the number of processes does not affect the performance. (I tried both 2-node and 4-node setting and got the same results).
(4) Applying the modded MLNX driver you provided does not change the performance.
(5) Making num_threads = 1 instead of 4 will make the Timely rate = [56.0, 56.0, 56.0, 56.0], and per thread tput becomes ~0.9 Mrps.

It would be great if you could give some hints on how to debug this issue. Thanks!

Best,
Yiwen

RoCE support

I see that ConnectX-3 NICs are supported in RoCE, but other NICs are coming soon. How do I select RoCE?

I am trying to set up softRoCE to work with eRPC, and while I do realize it's still under development, I've had some success with minimal changes:
vsbenas@2aa7d3c

I launched the hello_world app, and it seems the networking is working but there is an error on the client side once both are launched.
42:025353 WARNG: Rpc 0: Received connect response from [H: 192.168.122.103:31850, R: 0, S: XX] for session 0. Issue: Error [Invalid remote Rpc ID]. Segmentation fault

Failed tests in ctest never free hugepages

When ctest suite is run, some of the hugepages are never freed.

Use more flexible test running time

Right now, tests fail if they don't complete in kTestMaxEventLoopMs. Instead of having a deadline for the entire test, it's better to fail if we don't make progress for a long time.

Improving wait_for_rpc_resps_or_timeout and wait_for_sm_resps_or_timeout should be good enough.

run-tests-dpdk.sh failed.

Hello,
We tried to run eRPC on our machine but encountered some problems. Finally we were stuck at here:

Our environment variables are listed below:

Our environment variables are listed below:
Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ Network Connection
OS: CentOS
Kernel: x86_64 Linux 3.10.0-862.9.1.el7.x86_64
DPDK Version: 18.11

Could you please give us some advices? Thank you in advance!

API Question: Is it possible to broadcast without memcpy?

Hello Anuj,

Is it possible to enqueue the same request to multiple sessions using the same buffer space (i.e. "broadcast" by performing multiple unicasts)?

Allow me to elaborate.
From what i can tell, in order to perform a "broadcast", we must create multiple MsgBuffers -- one per each distinct RPC, because once a MsgBuffer is passed to eRPC (through enqueue_request()), it is no longer ours to pass it to enqueue_request() again (not until a response is received anyway, which would be too late).

Is it possible to enqueue multiple MsgBuffers, whose .buf point to the same location?
If not that leaves only one solution to broadcasting an RPC: creating one MsgBuffer per message and memcpying the .buf to all MsgBuffers. (But this seems unnecessary.)

Am I missing something?

Thank you very much,
vasilis

Add tests with different request and response message sizes

Currently, large_msg_test used identical sizes, which might miss some bugs.

this bug "rte_common.h: No such file or directory " also happened

Q for the design: How does ePRC handle the incast problem caused by the responses?

IIUC, ePRC only limits the number of requests to a responder by holding its credits. However, it does not limit the total outgoing requests and each request will get a response. Will it cause the incast problem when the massive responses arrive at the requester at the same time? If the problem matters, can eRPC handle it?

Looks forward for your reply.

Route resolution failure on Client + Segfault on Mellanox NIC

Hi,

OFED version 5.0.2
NIC Mellanox Connect X-5
OS Ubuntu 18
A tofino Switch does simple packet forwarding for us.

If I try with -DTransport=infiband and -DROCE=on I am able to build successfully and when I use the hello world app, on the client side I get the following error

Received connect response from [H: 192.168.1.4:31850, R: 0, S: XX] for session 0. Issue: Error [Routing resolution failure]

The server is receiving the initial connect packet from the client and then client segfaults and server prints the below statement in a loop. The error on the server is as follows.

Received connect request from [H: 192.168.1.3:31850, R: 0, S: 0]. Issue: Unable to resolve routing info [LID: 0, QPN: 449, GID interface ID 16601820732604482458, GID subnet prefix 33022]. Sending response.

In the README, you mentioned to use -Dtransport=raw for Mellanox NIC's. I was not able to build with that flag. Error Trace We want to use eRPC over ROCEv2 + DCQCN. We are okay with IB, unless you tell us otherwise. The RDMA devices are on on rdma link and ibdev2netdev.

Headroom for RDMA ROCE packets?

Why do ROCE eRPC packets have 42 bytes of headroom? I understand how this is used for other types of transport (e.g. "raw"), but for ROCE it just seems like a waste of space. For example, when I send a 48-byte RPC, there's a 14-byte eRPC header, plus 42 bytes of headroom (totaling 104 bytes):

I see that headroom is added when you compile with ROCE enabled:
https://github.com/erpc-io/eRPC/blob/master/CMakeLists.txt#L163

However, it looks like this headroom is never used in the infiniband transport. The only reference to headroom is an assertion here:
https://github.com/erpc-io/eRPC/blob/master/src/transport_impl/infiniband/ib_transport.cc#L31

I commented-out that assertion and re-compiled with set(CONFIG_HEADROOM 0). It doesn't seem to have affected functionality. In fact, it it seems to have dramatically reduced latency, especially for small payloads. This is the latency with 40 bytes of headroom:

$ ./scripts/do.sh 1 0
Installing modded drivers
do.sh: Launching process 1 on NUMA node 0
77:299277 WARNG: Modded driver unavailable. Performance will be low.
Process 1: Creating session to 10.0.1.98:31850.
Process 1: Session connected. Starting work.
write_size median_us 5th_us 99th_us 999th_us
32 5.1 5.0 5.9 8.1
64 5.1 5.0 5.9 8.8
128 5.2 5.1 6.0 8.7
256 5.2 5.1 6.1 8.8
512 5.4 5.2 6.2 8.8
1024 7.5 7.3 8.4 11.0

And after removing headroom:

$ ./scripts/do.sh 1 0                     
Installing modded drivers
do.sh: Launching process 1 on NUMA node 0
87:907285 WARNG: Modded driver unavailable. Performance will be low.
Process 1: Creating session to 10.0.1.98:31850.
Process 1: Session connected. Starting work.
write_size median_us 5th_us 99th_us 999th_us
32 4.1 4.0 5.1 7.7
64 4.6 4.5 5.6 8.8
128 4.6 4.5 5.6 8.6
256 4.7 4.6 5.7 8.2
512 4.8 4.7 5.8 8.8
1024 7.0 6.8 8.0 10.0

For 32-byte payloads, it reduced median latency by 1us!

Is this a bug? Should ROCE packets have these extra headroom bytes?

Compiling Error with MLNX_OFED_LINUX-5.3-1.0.0.1

Hi,

l am trying to compile the project with cmake . -DPERF=OFF -DTRANSPORT=raw; make. Although l have already installed OFED in version 5.3-1.0.0.1, l still met the following ibv related errors.The NIC l used is Mellanox ConnectX-5 MCX516-CCAT on ubuntu 18.04.

[  1%] Building CXX object CMakeFiles/erpc.dir/src/transport_impl/raw/raw_transport.cc.o
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc: In destructor ‘erpc::RawTransport::~RawTransport()’:
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:57:40: error: aggregate ‘erpc::RawTransport::~RawTransport()::ibv_exp_release_intf_params rel_intf_params’ has incomplete type and cannot be defined
     struct ibv_exp_release_intf_params rel_intf_params;
                                        ^~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:60:9: error: ‘ibv_exp_release_intf’ was not declared in this scope
         ibv_exp_release_intf(resolve.ib_ctx, wq_family, &rel_intf_params) == 0,
         ^~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:60:9: note: suggested alternative: ‘ibv_exp_release_intf_params’
         ibv_exp_release_intf(resolve.ib_ctx, wq_family, &rel_intf_params) == 0,
         ^~~~~~~~~~~~~~~~~~~~
         ibv_exp_release_intf_params
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:63:17: error: ‘ibv_exp_destroy_flow’ was not declared in this scope
     exit_assert(ibv_exp_destroy_flow(recv_flow) == 0,
                 ^~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:63:17: note: suggested alternative: ‘ibv_destroy_flow’
     exit_assert(ibv_exp_destroy_flow(recv_flow) == 0,
                 ^~~~~~~~~~~~~~~~~~~~
                 ibv_destroy_flow
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:69:17: error: ‘ibv_exp_destroy_rwq_ind_table’ was not declared in this scope
     exit_assert(ibv_exp_destroy_rwq_ind_table(ind_tbl) == 0,
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:69:17: note: suggested alternative: ‘ibv_destroy_rwq_ind_table’
     exit_assert(ibv_exp_destroy_rwq_ind_table(ind_tbl) == 0,
                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                 ibv_destroy_rwq_ind_table
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:72:17: error: ‘ibv_exp_destroy_wq’ was not declared in this scope
     exit_assert(ibv_exp_destroy_wq(wq) == 0, "Failed to destroy WQ");
                 ^~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:72:17: note: suggested alternative: ‘ibv_destroy_wq’
     exit_assert(ibv_exp_destroy_wq(wq) == 0, "Failed to destroy WQ");
                 ^~~~~~~~~~~~~~~~~~
                 ibv_destroy_wq
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:74:17: error: ‘ibv_exp_destroy_flow’ was not declared in this scope
     exit_assert(ibv_exp_destroy_flow(recv_flow) == 0,
                 ^~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:74:17: note: suggested alternative: ‘ibv_destroy_flow’
     exit_assert(ibv_exp_destroy_flow(recv_flow) == 0,
                 ^~~~~~~~~~~~~~~~~~~~
                 ibv_destroy_flow
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc: In member function ‘void erpc::RawTransport::init_basic_qp()’:
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:124:31: error: aggregate ‘erpc::RawTransport::init_basic_qp()::ibv_exp_cq_init_attr cq_init_attr’ has incomplete type and cannot be defined
   struct ibv_exp_cq_init_attr cq_init_attr;
                               ^~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:126:13: error: ‘ibv_exp_create_cq’ was not declared in this scope
   send_cq = ibv_exp_create_cq(resolve.ib_ctx, kSQDepth, nullptr, nullptr, 0,
             ^~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:126:13: note: suggested alternative: ‘ibv_create_cq’
   send_cq = ibv_exp_create_cq(resolve.ib_ctx, kSQDepth, nullptr, nullptr, 0,
             ^~~~~~~~~~~~~~~~~
             ibv_create_cq
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:137:31: error: aggregate ‘erpc::RawTransport::init_basic_qp()::ibv_exp_qp_init_attr qp_init_attr’ has incomplete type and cannot be defined
   struct ibv_exp_qp_init_attr qp_init_attr;
                               ^~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:139:28: error: ‘IBV_EXP_QP_INIT_ATTR_PD’ was not declared in this scope
   qp_init_attr.comp_mask = IBV_EXP_QP_INIT_ATTR_PD;
                            ^~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:139:28: note: suggested alternative: ‘IBV_QP_INIT_ATTR_PD’
   qp_init_attr.comp_mask = IBV_EXP_QP_INIT_ATTR_PD;
                            ^~~~~~~~~~~~~~~~~~~~~~~
                            IBV_QP_INIT_ATTR_PD
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:151:8: error: ‘ibv_exp_create_qp’ was not declared in this scope
   qp = ibv_exp_create_qp(resolve.ib_ctx, &qp_init_attr);
        ^~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:151:8: note: suggested alternative: ‘ibv_create_qp’
   qp = ibv_exp_create_qp(resolve.ib_ctx, &qp_init_attr);
        ^~~~~~~~~~~~~~~~~
        ibv_create_qp
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:154:26: error: aggregate ‘erpc::RawTransport::init_basic_qp()::ibv_exp_qp_attr qp_attr’ has incomplete type and cannot be defined
   struct ibv_exp_qp_attr qp_attr;
                          ^~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:158:13: error: ‘ibv_exp_modify_qp’ was not declared in this scope
   rt_assert(ibv_exp_modify_qp(qp, &qp_attr, IBV_QP_STATE | IBV_QP_PORT) == 0);
             ^~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:158:13: note: suggested alternative: ‘ibv_modify_qp’
   rt_assert(ibv_exp_modify_qp(qp, &qp_attr, IBV_QP_STATE | IBV_QP_PORT) == 0);
             ^~~~~~~~~~~~~~~~~
             ibv_modify_qp
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc: In member function ‘void erpc::RawTransport::init_mp_recv_qp()’:
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:174:31: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_cq_init_attr cq_init_attr’ has incomplete type and cannot be defined
   struct ibv_exp_cq_init_attr cq_init_attr;
                               ^~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:176:13: error: ‘ibv_exp_create_cq’ was not declared in this scope
   recv_cq = ibv_exp_create_cq(resolve.ib_ctx, kRecvCQDepth / 2, nullptr,
             ^~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:176:13: note: suggested alternative: ‘ibv_create_cq’
   recv_cq = ibv_exp_create_cq(resolve.ib_ctx, kRecvCQDepth / 2, nullptr,
             ^~~~~~~~~~~~~~~~~
             ibv_create_cq
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:181:26: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_cq_attr cq_attr’ has incomplete type and cannot be defined
   struct ibv_exp_cq_attr cq_attr;
                          ^~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:183:23: error: ‘IBV_EXP_CQ_ATTR_CQ_CAP_FLAGS’ was not declared in this scope
   cq_attr.comp_mask = IBV_EXP_CQ_ATTR_CQ_CAP_FLAGS;
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:183:23: note: suggested alternative: ‘IBV_QP_INIT_ATTR_CREATE_FLAGS’
   cq_attr.comp_mask = IBV_EXP_CQ_ATTR_CQ_CAP_FLAGS;
                       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
                       IBV_QP_INIT_ATTR_CREATE_FLAGS
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:184:26: error: ‘IBV_EXP_CQ_IGNORE_OVERRUN’ was not declared in this scope
   cq_attr.cq_cap_flags = IBV_EXP_CQ_IGNORE_OVERRUN;
                          ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:184:26: note: suggested alternative: ‘IBV_CREATE_CQ_ATTR_IGNORE_OVERRUN’
   cq_attr.cq_cap_flags = IBV_EXP_CQ_IGNORE_OVERRUN;
                          ^~~~~~~~~~~~~~~~~~~~~~~~~
                          IBV_CREATE_CQ_ATTR_IGNORE_OVERRUN
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:185:50: error: ‘IBV_EXP_CQ_CAP_FLAGS’ was not declared in this scope
   rt_assert(ibv_exp_modify_cq(recv_cq, &cq_attr, IBV_EXP_CQ_CAP_FLAGS) == 0);
                                                  ^~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:185:50: note: suggested alternative: ‘IBV_QP_ACCESS_FLAGS’
   rt_assert(ibv_exp_modify_cq(recv_cq, &cq_attr, IBV_EXP_CQ_CAP_FLAGS) == 0);
                                                  ^~~~~~~~~~~~~~~~~~~~
                                                  IBV_QP_ACCESS_FLAGS
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:185:13: error: ‘ibv_exp_modify_cq’ was not declared in this scope
   rt_assert(ibv_exp_modify_cq(recv_cq, &cq_attr, IBV_EXP_CQ_CAP_FLAGS) == 0);
             ^~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:185:13: note: suggested alternative: ‘ibv_modify_cq’
   rt_assert(ibv_exp_modify_cq(recv_cq, &cq_attr, IBV_EXP_CQ_CAP_FLAGS) == 0);
             ^~~~~~~~~~~~~~~~~
             ibv_modify_cq
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:187:31: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_wq_init_attr wq_init_attr’ has incomplete type and cannot be defined
   struct ibv_exp_wq_init_attr wq_init_attr;
                               ^~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:190:26: error: ‘IBV_EXP_WQT_RQ’ was not declared in this scope
   wq_init_attr.wq_type = IBV_EXP_WQT_RQ;
                          ^~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:190:26: note: suggested alternative: ‘IBV_WQT_RQ’
   wq_init_attr.wq_type = IBV_EXP_WQT_RQ;
                          ^~~~~~~~~~~~~~
                          IBV_WQT_RQ
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:196:29: error: ‘IBV_EXP_CREATE_WQ_MP_RQ’ was not declared in this scope
   wq_init_attr.comp_mask |= IBV_EXP_CREATE_WQ_MP_RQ;
                             ^~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:196:29: note: suggested alternative: ‘IBV_QP_RATE_LIMIT’
   wq_init_attr.comp_mask |= IBV_EXP_CREATE_WQ_MP_RQ;
                             ^~~~~~~~~~~~~~~~~~~~~~~
                             IBV_QP_RATE_LIMIT
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:197:34: error: ‘IBV_EXP_MP_RQ_NO_SHIFT’ was not declared in this scope
   wq_init_attr.mp_rq.use_shift = IBV_EXP_MP_RQ_NO_SHIFT;
                                  ^~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:197:34: note: suggested alternative: ‘IBV_WC_IP_CSUM_OK_SHIFT’
   wq_init_attr.mp_rq.use_shift = IBV_EXP_MP_RQ_NO_SHIFT;
                                  ^~~~~~~~~~~~~~~~~~~~~~
                                  IBV_WC_IP_CSUM_OK_SHIFT
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:200:8: error: ‘ibv_exp_create_wq’ was not declared in this scope
   wq = ibv_exp_create_wq(resolve.ib_ctx, &wq_init_attr);
        ^~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:200:8: note: suggested alternative: ‘ibv_create_wq’
   wq = ibv_exp_create_wq(resolve.ib_ctx, &wq_init_attr);
        ^~~~~~~~~~~~~~~~~
        ibv_create_wq
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:204:26: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_wq_attr wq_attr’ has incomplete type and cannot be defined
   struct ibv_exp_wq_attr wq_attr;
                          ^~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:206:23: error: ‘IBV_EXP_WQ_ATTR_STATE’ was not declared in this scope
   wq_attr.attr_mask = IBV_EXP_WQ_ATTR_STATE;
                       ^~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:206:23: note: suggested alternative: ‘IBV_WQ_ATTR_STATE’
   wq_attr.attr_mask = IBV_EXP_WQ_ATTR_STATE;
                       ^~~~~~~~~~~~~~~~~~~~~
                       IBV_WQ_ATTR_STATE
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:207:22: error: ‘IBV_EXP_WQS_RDY’ was not declared in this scope
   wq_attr.wq_state = IBV_EXP_WQS_RDY;
                      ^~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:207:22: note: suggested alternative: ‘IBV_WQS_RDY’
   wq_attr.wq_state = IBV_EXP_WQS_RDY;
                      ^~~~~~~~~~~~~~~
                      IBV_WQS_RDY
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:208:13: error: ‘ibv_exp_modify_wq’ was not declared in this scope
   rt_assert(ibv_exp_modify_wq(wq, &wq_attr) == 0, "Failed to ready WQ");
             ^~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:208:13: note: suggested alternative: ‘ibv_modify_wq’
   rt_assert(ibv_exp_modify_wq(wq, &wq_attr) == 0, "Failed to ready WQ");
             ^~~~~~~~~~~~~~~~~
             ibv_modify_wq
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:211:8: error: use of enum ‘ibv_exp_query_intf_status’ without previous declaration
   enum ibv_exp_query_intf_status intf_status = IBV_EXP_INTF_STAT_OK;
        ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:211:48: error: ‘IBV_EXP_INTF_STAT_OK’ was not declared in this scope
   enum ibv_exp_query_intf_status intf_status = IBV_EXP_INTF_STAT_OK;
                                                ^~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:211:48: note: suggested alternative: ‘IBV_QP_CUR_STATE’
   enum ibv_exp_query_intf_status intf_status = IBV_EXP_INTF_STAT_OK;
                                                ^~~~~~~~~~~~~~~~~~~~
                                                IBV_QP_CUR_STATE
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:212:36: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_query_intf_params query_intf_params’ has incomplete type and cannot be defined
   struct ibv_exp_query_intf_params query_intf_params;
                                    ^~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:214:34: error: ‘IBV_EXP_INTF_GLOBAL’ was not declared in this scope
   query_intf_params.intf_scope = IBV_EXP_INTF_GLOBAL;
                                  ^~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:214:34: note: suggested alternative: ‘IBV_ATOMIC_GLOB’
   query_intf_params.intf_scope = IBV_EXP_INTF_GLOBAL;
                                  ^~~~~~~~~~~~~~~~~~~
                                  IBV_ATOMIC_GLOB
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:215:28: error: ‘IBV_EXP_INTF_WQ’ was not declared in this scope
   query_intf_params.intf = IBV_EXP_INTF_WQ;
                            ^~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:218:7: error: ‘ibv_exp_query_intf’ was not declared in this scope
       ibv_exp_query_intf(resolve.ib_ctx, &query_intf_params, &intf_status));
       ^~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:218:7: note: suggested alternative: ‘ibv_exp_peer_buf’
       ibv_exp_query_intf(resolve.ib_ctx, &query_intf_params, &intf_status));
       ^~~~~~~~~~~~~~~~~~
       ibv_exp_peer_buf
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:222:42: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_rwq_ind_table_init_attr rwq_ind_table_init_attr’ has incomplete type and cannot be defined
   struct ibv_exp_rwq_ind_table_init_attr rwq_ind_table_init_attr;
                                          ^~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:229:7: error: ‘ibv_exp_create_rwq_ind_table’ was not declared in this scope
       ibv_exp_create_rwq_ind_table(resolve.ib_ctx, &rwq_ind_table_init_attr);
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:229:7: note: suggested alternative: ‘ibv_create_rwq_ind_table’
       ibv_exp_create_rwq_ind_table(resolve.ib_ctx, &rwq_ind_table_init_attr);
       ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
       ibv_create_rwq_ind_table
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:241:31: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_rx_hash_conf rx_hash_conf’ has incomplete type and cannot be defined
   struct ibv_exp_rx_hash_conf rx_hash_conf;
                               ^~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:243:35: error: ‘IBV_EXP_RX_HASH_FUNC_TOEPLITZ’ was not declared in this scope
   rx_hash_conf.rx_hash_function = IBV_EXP_RX_HASH_FUNC_TOEPLITZ;
                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:243:35: note: suggested alternative: ‘IBV_RX_HASH_FUNC_TOEPLITZ’
   rx_hash_conf.rx_hash_function = IBV_EXP_RX_HASH_FUNC_TOEPLITZ;
                                   ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                   IBV_RX_HASH_FUNC_TOEPLITZ
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:246:38: error: ‘IBV_EXP_RX_HASH_DST_PORT_UDP’ was not declared in this scope
   rx_hash_conf.rx_hash_fields_mask = IBV_EXP_RX_HASH_DST_PORT_UDP;
                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:246:38: note: suggested alternative: ‘IBV_RX_HASH_DST_PORT_UDP’
   rx_hash_conf.rx_hash_fields_mask = IBV_EXP_RX_HASH_DST_PORT_UDP;
                                      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
                                      IBV_RX_HASH_DST_PORT_UDP
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:249:31: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_qp_init_attr qp_init_attr’ has incomplete type and cannot be defined
   struct ibv_exp_qp_init_attr qp_init_attr;
                               ^~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:251:28: error: ‘IBV_EXP_QP_INIT_ATTR_CREATE_FLAGS’ was not declared in this scope
   qp_init_attr.comp_mask = IBV_EXP_QP_INIT_ATTR_CREATE_FLAGS |
                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:251:28: note: suggested alternative: ‘IBV_QP_INIT_ATTR_CREATE_FLAGS’
   qp_init_attr.comp_mask = IBV_EXP_QP_INIT_ATTR_CREATE_FLAGS |
                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
                            IBV_QP_INIT_ATTR_CREATE_FLAGS
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:252:28: error: ‘IBV_EXP_QP_INIT_ATTR_PD’ was not declared in this scope
                            IBV_EXP_QP_INIT_ATTR_PD |
                            ^~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:252:28: note: suggested alternative: ‘IBV_QP_INIT_ATTR_PD’
                            IBV_EXP_QP_INIT_ATTR_PD |
                            ^~~~~~~~~~~~~~~~~~~~~~~
                            IBV_QP_INIT_ATTR_PD
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:253:28: error: ‘IBV_EXP_QP_INIT_ATTR_RX_HASH’ was not declared in this scope
                            IBV_EXP_QP_INIT_ATTR_RX_HASH;
                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:253:28: note: suggested alternative: ‘IBV_QP_INIT_ATTR_RX_HASH’
                            IBV_EXP_QP_INIT_ATTR_RX_HASH;
                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
                            IBV_QP_INIT_ATTR_RX_HASH
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:259:16: error: ‘ibv_exp_create_qp’ was not declared in this scope
   mp_recv_qp = ibv_exp_create_qp(resolve.ib_ctx, &qp_init_attr);
                ^~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:259:16: note: suggested alternative: ‘ibv_create_qp’
   mp_recv_qp = ibv_exp_create_qp(resolve.ib_ctx, &qp_init_attr);
                ^~~~~~~~~~~~~~~~~
                ibv_create_qp
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc: In member function ‘void erpc::RawTransport::install_flow_rule()’:
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:273:14: error: ‘ibv_exp_flow_attr’ was not declared in this scope
       sizeof(ibv_exp_flow_attr) + sizeof(ibv_exp_flow_spec_eth) +
              ^~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:273:14: note: suggested alternative: ‘ibv_flow_attr’
       sizeof(ibv_exp_flow_attr) + sizeof(ibv_exp_flow_spec_eth) +
              ^~~~~~~~~~~~~~~~~
              ibv_flow_attr
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:273:42: error: ‘ibv_exp_flow_spec_eth’ was not declared in this scope
       sizeof(ibv_exp_flow_attr) + sizeof(ibv_exp_flow_spec_eth) +
                                          ^~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:273:42: note: suggested alternative: ‘ibv_flow_spec_eth’
       sizeof(ibv_exp_flow_attr) + sizeof(ibv_exp_flow_spec_eth) +
                                          ^~~~~~~~~~~~~~~~~~~~~
                                          ibv_flow_spec_eth
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:274:14: error: ‘ibv_exp_flow_spec_ipv4_ext’ was not declared in this scope
       sizeof(ibv_exp_flow_spec_ipv4_ext) + sizeof(ibv_exp_flow_spec_tcp_udp);
              ^~~~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:274:14: note: suggested alternative: ‘ibv_flow_spec_ipv4_ext’
       sizeof(ibv_exp_flow_spec_ipv4_ext) + sizeof(ibv_exp_flow_spec_tcp_udp);
              ^~~~~~~~~~~~~~~~~~~~~~~~~~
              ibv_flow_spec_ipv4_ext
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:274:51: error: ‘ibv_exp_flow_spec_tcp_udp’ was not declared in this scope
       sizeof(ibv_exp_flow_spec_ipv4_ext) + sizeof(ibv_exp_flow_spec_tcp_udp);
                                                   ^~~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:274:51: note: suggested alternative: ‘ibv_flow_spec_tcp_udp’
       sizeof(ibv_exp_flow_spec_ipv4_ext) + sizeof(ibv_exp_flow_spec_tcp_udp);
                                                   ^~~~~~~~~~~~~~~~~~~~~~~~~
                                                   ibv_flow_spec_tcp_udp
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:276:28: error: size of array ‘flow_rule’ is not an integral constant-expression
   uint8_t flow_rule[rule_sz];
                            ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:280:63: error: expected ‘>’ before ‘*’ token
   auto *flow_attr = reinterpret_cast<struct ibv_exp_flow_attr *>(flow_rule);
                                                               ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:280:63: error: expected ‘(’ before ‘*’ token
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:280:64: error: expected primary-expression before ‘>’ token
   auto *flow_attr = reinterpret_cast<struct ibv_exp_flow_attr *>(flow_rule);
                                                                ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:280:76: error: expected ‘)’ before ‘;’ token
   auto *flow_attr = reinterpret_cast<struct ibv_exp_flow_attr *>(flow_rule);
                                                                            ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:281:21: error: ‘IBV_EXP_FLOW_ATTR_NORMAL’ was not declared in this scope
   flow_attr->type = IBV_EXP_FLOW_ATTR_NORMAL;
                     ^~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:281:21: note: suggested alternative: ‘IBV_FLOW_ATTR_NORMAL’
   flow_attr->type = IBV_EXP_FLOW_ATTR_NORMAL;
                     ^~~~~~~~~~~~~~~~~~~~~~~~
                     IBV_FLOW_ATTR_NORMAL
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:291:66: error: expected ‘>’ before ‘*’ token
   auto *eth_spec = reinterpret_cast<struct ibv_exp_flow_spec_eth *>(buf);
                                                                  ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:291:66: error: expected ‘(’ before ‘*’ token
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:291:67: error: expected primary-expression before ‘>’ token
   auto *eth_spec = reinterpret_cast<struct ibv_exp_flow_spec_eth *>(buf);
                                                                   ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:291:73: error: expected ‘)’ before ‘;’ token
   auto *eth_spec = reinterpret_cast<struct ibv_exp_flow_spec_eth *>(buf);
                                                                         ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:292:20: error: ‘IBV_EXP_FLOW_SPEC_ETH’ was not declared in this scope
   eth_spec->type = IBV_EXP_FLOW_SPEC_ETH;
                    ^~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:292:20: note: suggested alternative: ‘IBV_FLOW_SPEC_ETH’
   eth_spec->type = IBV_EXP_FLOW_SPEC_ETH;
                    ^~~~~~~~~~~~~~~~~~~~~
                    IBV_FLOW_SPEC_ETH
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:299:72: error: expected ‘>’ before ‘*’ token
   auto *spec_ipv4 = reinterpret_cast<struct ibv_exp_flow_spec_ipv4_ext *>(buf);
                                                                        ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:299:72: error: expected ‘(’ before ‘*’ token
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:299:73: error: expected primary-expression before ‘>’ token
   auto *spec_ipv4 = reinterpret_cast<struct ibv_exp_flow_spec_ipv4_ext *>(buf);
                                                                         ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:299:79: error: expected ‘)’ before ‘;’ token
   auto *spec_ipv4 = reinterpret_cast<struct ibv_exp_flow_spec_ipv4_ext *>(buf);
                                                                               ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:300:21: error: ‘IBV_EXP_FLOW_SPEC_IPV4_EXT’ was not declared in this scope
   spec_ipv4->type = IBV_EXP_FLOW_SPEC_IPV4_EXT;
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:300:21: note: suggested alternative: ‘IBV_FLOW_SPEC_IPV4_EXT’
   spec_ipv4->type = IBV_EXP_FLOW_SPEC_IPV4_EXT;
                     ^~~~~~~~~~~~~~~~~~~~~~~~~~
                     IBV_FLOW_SPEC_IPV4_EXT
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:305:70: error: expected ‘>’ before ‘*’ token
   auto *udp_spec = reinterpret_cast<struct ibv_exp_flow_spec_tcp_udp *>(buf);
                                                                      ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:305:70: error: expected ‘(’ before ‘*’ token
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:305:71: error: expected primary-expression before ‘>’ token
   auto *udp_spec = reinterpret_cast<struct ibv_exp_flow_spec_tcp_udp *>(buf);
                                                                       ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:305:77: error: expected ‘)’ before ‘;’ token
   auto *udp_spec = reinterpret_cast<struct ibv_exp_flow_spec_tcp_udp *>(buf);
                                                                             ^
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:306:20: error: ‘IBV_EXP_FLOW_SPEC_UDP’ was not declared in this scope
   udp_spec->type = IBV_EXP_FLOW_SPEC_UDP;
                    ^~~~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:306:20: note: suggested alternative: ‘IBV_FLOW_SPEC_UDP’
   udp_spec->type = IBV_EXP_FLOW_SPEC_UDP;
                    ^~~~~~~~~~~~~~~~~~~~~
                    IBV_FLOW_SPEC_UDP
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:311:15: error: ‘ibv_exp_create_flow’ was not declared in this scope
   recv_flow = ibv_exp_create_flow(qp_for_flow, flow_attr);
               ^~~~~~~~~~~~~~~~~~~
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:311:15: note: suggested alternative: ‘ibv_create_flow’
   recv_flow = ibv_exp_create_flow(qp_for_flow, flow_attr);
               ^~~~~~~~~~~~~~~~~~~
               ibv_create_flow
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc: In member function ‘void erpc::RawTransport::init_recvs(uint8_t**)’:
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:397:16: error: invalid use of incomplete type ‘struct erpc::ibv_exp_wq_family’
       wq_family->recv_burst(wq, &mp_recv_sge[i], 1);
                ^~
In file included from /home/hjh/eRPC/src/transport_impl/raw/raw_transport.cc:6:0:
/home/hjh/eRPC/src/transport_impl/raw/raw_transport.h:202:10: note: forward declaration of ‘struct erpc::ibv_exp_wq_family’
   struct ibv_exp_wq_family *wq_family;
          ^~~~~~~~~~~~~~~~~
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]

Is there any version related problem in OFED or something else?

Look forward for you comments.
Thanks.

How does this code works well?

Building with QLogic driver on ubuntu16.04

Hi,
I was trying to build the project using QLogic HCAs.
By setting -DTRANSPORT=infiniband, building process reported the following error:
eRPC/src/transport_impl/raw/mlx5_defs.h:10:34: fatal error: infiniband/verbs_exp.h: No such file or directory
It seems verbs_exp.h is mellanox specific header and I don't have it in my QLogic environment.

Thanks,
Zhang

"Ethdev configuration error: Invalid argument" error

Hi,

I'm getting an error:
Most tests in scripts/run-tests-dpkg.sh:

C++ exception with description "Ethdev configuration error: Invalid argument" thrown in the test fixture's constructor.

I get the same error in hello_world app as well.

Maybe I messed something up with the network configuration, but I can't find any info online about this error.

ARP packet gets through DPDK filter

On X710 NICs, this ARP packet got through the filter:

15:300779 TRACE:   Transport: RX pkthdr = [type REQ, dsn 0, reqn 0, pktn 0, msz 0]. Frame = [ETH: dst ff:ff:ff:ff:ff:ff, src 0:50:56:a1:28:a7, eth_type 2054], [IPv4: ihl 0, version 0, ecn 1, tot_len 2048, id 1540, frag_off 1, ttl 0, protocol 80, check 41302, src IP 40.167.10.11, dst IP 6.71.0.0], [UDP: src_port 0, dst_port 0, len 2571, check 19718].

We need a filtering test.

Naming convention: the meaning of the suffix _st in function name?

What does the suffix _st in function name mean? There are many functions with the above suffix in eRPC.

Questions about the latency app

Hi Anuj,

I run into a few problems when running the latency benchmark on CloudLab xl170 machines.

The latency numbers I get seem to be lower than numbers reported in the paper:

29:082809 WARNG: Installing flow rule for Rpc 0. NUMA node = 0. Flow RX UDP port = 31882.
29:091714 WARNG: RawTransport created for Rpc ID 0. Device mlx5_3/ens1f1, port 1. IPv4 10.10.1.3, MAC 9c:dc:71:5b:42:81. Datapath UDP port 31882.
Process 1: Creating session to rc01.erpc.ramcloud-pg0.utah.cloudlab.us:31850.
Process 1: Session connected. Starting work.
write_size median_us 5th_us 99th_us 999th_us
64 4.5 4.4 5.0 5.7
128 4.5 4.4 4.9 5.6
256 4.6 4.5 5.0 5.6
512 4.8 4.7 5.1 6.1
1024 6.1 6.0 6.5 7.2
64 4.5 4.4 5.0 5.7
128 4.5 4.4 4.9 5.7
256 4.6 4.5 5.0 5.9
512 4.8 4.7 5.0 5.8
1024 6.1 6.0 6.5 7.3
64 4.5 4.4 5.0 5.7
128 4.5 4.4 4.9 5.7
256 4.6 4.5 5.0 5.9
512 4.8 4.7 5.0 6.0
1024 6.1 5.9 6.5 7.4
64 4.5 4.4 5.0 5.6
128 4.5 4.4 5.0 5.5
256 4.6 4.5 5.0 5.7
512 4.8 4.7 5.0 6.0
1024 6.1 5.9 6.5 7.4

The latency for small message is about 0.8us higher than 3.7us reported in the paper. I build eRPC with cmake . -DPERF=ON -DLOG_LEVEL=warn -DTRANSPORT=raw. Is it because I am not using your optimized drivers? How can I use them?

scripts/proc-out.sh seems to be broken:

yilongl@rcnfs:/shome/eRPC$ scripts/proc-out.sh
autorun: app = latency, test ms = 20000, num_processes = 2
proc-out: Fetching latency_stats_0 from rc01.erpc.ramcloud-pg0.utah.cloudlab.us
proc-out: Fetching latency_stats_1 from rc02.erpc.ramcloud-pg0.utah.cloudlab.us
proc-out: Finished fetching files.
cat: '/tmp/latency_proc/*': No such file or directory
proc-out: Header = []
proc-out: Detected 0 columns
ls: cannot access '/tmp/latency_proc/*': No such file or directory
proc-out: Processed files = 0, ignored files = 0

The files generated by latency are /tmp/latency_{err,out}-{0,1}.

When I use scripts/run-all.sh to run the experiment, I usually get message like run-all: latency still running on XXX at the end. Is it OK to pkill these processes in the script? Are they still doing anything useful?

Additional info:

Machine: CloudLab xl170
OS: Ubuntu 18.04
MLNX OFED: 4.6-1.0.1.1

Thank you.

Improving performance using LTO and drivers

I'm trying to improve performance of my eRPC application and would like to try using LTO instead of compiling eRPC as a library. Are there any instructions how to do so?
Currently I'm using -lerpc when compiling my program.

Also, I'm getting a warning

WARNG: Modded driver unavailable. Performance will be low.

I assume using the provided drivers will increase the performance, but I can't find any guides how to do that as well. Could you help me out with this? I'm sure it will be helpful to other users as well.

NIC model: 56Gb Infiniband NIC (Mellanox MCX455A-FCAT PCIe-gen3 x16)
Mellanox OFED or DPDK version: MLNX_OFED_LINUX-4.5-1.0.1.0-ubuntu18.04-x86_64
Operating system: Ubuntu 18.04

Continuation-Function and Request-Handlers

Hi again,

If an application thread enqueues a request, will the continuation-function for this specific request be executed by the same specific application thread or can be executed any other application thread running? (for simplification erpc-background threads are disabled).

Likewise, in the remote machine, the thread that will execute the request_handler function, will be the thread that is "connected" (through Remote ID) with the sender or any thread that might be available?

Thanks,
--Dimitra

Clarify the requirements for compiler

Hi Anuj, could you clarify which version of gcc is supported by eRPC? I am using gcc 7.4.0 that comes with Ubuntu 18.04 and it doesn't recognize options like -Wno-nested-anon-types and -Wno-keyword-macro. Also, I got an unused variable error (which I fixed with (void)si;) at:

eRPC/src/rpc_impl/rpc_disconnect_handlers.cc

Line 39 in dff4589

const auto &si = sslot.server_info;

app/small_rpc_tput failed with modified libmlx5 driver

Hi Anuj,

I was trying the modified mlx5 driver. I'm running OFED 4.4.
While linked with this modified driver, I got: eRPC: Fatal error. Bad wc status 19.

Compared with driver/4.2/libmlx4, I found you missed some code in both driver/4.4/libmlx5 and driver/4.2/libmlx5. My question is: have you fully tested the mlx5 modified driver? Or should I start look into mlx4 one to port anything that is missing?

Can I run erpc in docker ?

Hi,
When I ran erpc in dcoker , I got an error "Failed to open dev 0".
And this is my docker setup cmd: "sudo docker run -ti -d --name=xxx --net=host -v /xxx:/xxx -v /dev:/dev docker-image /bin/bash "
Thanks.

ctest all fail

Hi,
I build eRPC with

 ./scripts/packages/ubuntu18/required.sh
cmake . -DPERF=OFF -DTRANSPORT=infiniband
make -j

But it faild all test in ctest

The following tests FAILED:
          1 - create_session_test (Child aborted)
          2 - destroy_session_test (Child aborted)
          3 - small_msg_test (Child aborted)
          4 - large_msg_test (Child aborted)
          5 - req_in_cont_func_test (Child aborted)
          6 - req_in_req_func_test (Child aborted)
          7 - packet_loss_test (Child aborted)
          8 - multi_process_test (Child aborted)
          9 - rpc_sm_test (Child aborted)
         10 - rpc_list_test (Failed)
         11 - rpc_req_test (Child aborted)
         12 - rpc_resp_test (Child aborted)
         13 - rpc_cr_test (Failed)
         14 - rpc_rfr_test (Failed)
         15 - rpc_kick_test (Child aborted)

Do anybody know what cause this?

I also tried to build&run hello_work, but got error from running sudo ./hello_world/server


81:654263 WARNG: eRPC Nexus: Testing enabled. Perf will be low.
terminate called after throwing an instance of 'std::runtime_error'
  what():  Failed to get device list

Thanks for any hints.

Compiling Error on cluster with no DPDK library

I'm trying to run your eRPC on the CloudLab cluster. After I run:

 cmake . -DPERF=OFF -DTRANSPORT=infiniband
 make

I got an error message:

In file included from /users/jingzy/eRPC/src/cc/timing_wheel.h:19:0,
                 from /users/jingzy/eRPC/src/session.h:8,
                 from /users/jingzy/eRPC/src/nexus.h:6,
                 from /users/jingzy/eRPC/src/nexus_impl/nexus.cc:1:
/users/jingzy/eRPC/src/transport_impl/dpdk/dpdk_transport.h:12:10: fatal error: rte_common.h: No such file or directory
 #include <rte_common.h>

I guess it might be the reason that my cluster doesn't install DPDK's library. But since I'm running on the InfiniBand mode, is there any way to fix this without installing that library?

By the way, I notice that an autorun_app_file should be used to indicate the app that needs to be compiled together with. In you CMakeList, the .cc should have the same name as its directory. How can I do this for app/hello since I find no hello.cc inside hello?

Can we re-create the same RPC object with the same Remote id?

Hi everyone,

When using eRPC + dpdk:

Let's assume a session between two machines (following the hello world example).

If this session is lost or the client-server gets disconnected, it is possible to re-create it using the same Remote ID?

It seems that it is possible to re-create it and re-connect to the server-machine but when an enqueue_operation() is executed, a seg-fault occurs.

Based on gdb-analysis the error lies ''somewhere'' in dpdk code.

#0 0x00000000006eb3a4 in common_ring_mc_dequeue ()
#1 0x0000000000811d65 in i40e_recv_scattered_pkts ()
#2 0x0000000000a2c674 in rte_eth_rx_burst (nb_pkts=32, rx_pkts=0x7fffffffafb0, queue_id=0, port_id=0) at /home/eRPC/dpdk/x86_64-native-linuxapp-gcc/include/rte_ethdev.h:4097
#3 erpc::DpdkTransport::rx_burst (this=0xf2c870) at /home/eRPC/src/transport_impl/dpdk/dpdk_transport_datapath.cc:103

Is such an action normally supported by eRPC?

-- Thanks,
Dimitra

Improve exception safety with smart pointers

Would you like to wrap any pointer data members with the class template “std::unique_ptr”?

Update candidates:

Rpc
RpcTest

Unable to achieve 10M RPCs/sec with a single core on CloudLab xl170

Here is how I configure the build:

yilongl@rcnfs:/shome/eRPC$ cmake . -DPERF=ON -DLOG_LEVEL=warn -DTRANSPORT=raw
-- Compilation optimized for performance.
-- Profile-guided optimization is disabled.
-- Logging level = warn.
-- Debugging is disabled.
-- Testing is disabled, so tests may fail.
-- LTO is enabled. eRPC library won't be compiled.
-- Boost version: 1.65.1
-- Boost include directory = /usr/include
-- Selected transport = raw.
-- Compiling app = small_rpc_tput
-- pmem library not found
-- Configuring done
-- Generating done
-- Build files have been written to: /shome/eRPC

And here is the config file of small_rpc_tput:

yilongl@rcnfs:/shome/eRPC$ cat apps/small_rpc_tput/config 
--test_ms 20000
--sm_verbose 0
--batch_size 1  
--concurrency 60
--msg_size 32
--num_processes 2
--num_threads 1
--numa_0_ports 0
--numa_1_ports 1,3

I can only get ~1.5M RPCs/sec using one CPU core. Here is the output from one server:

yilongl@rc02:/shome/eRPC$ scripts/do.sh $((`hostname -s | grep -o "[0-9]*"` - 1)) 0
Installing modded drivers
do.sh: Launching process 1 on NUMA node 0
Writing stats to file /tmp/small_rpc_tput_stats_1
40:275278 WARNG: Installing flow rule for Rpc 0. NUMA node = 0. Flow RX UDP port = 31882.
40:284131 WARNG: RawTransport created for Rpc ID 0. Device mlx5_3/ens1f1, port 1. IPv4 10.10.1.3, MAC 9c:dc:71:5b:42:81. Datapath UDP port 31882.
Process 1, thread 0: All sessions connected. Starting work.
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1455K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1455K resps, 1455K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.456 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1455K resps, 1455K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.454 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1455K resps, 1455K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.454 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1455K resps, 1455K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.453 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1453K resps, 1453K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.455 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1454K resps, 1454K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].
Process 1, thread 0: 1.454 Mrps, re_tx = 0, still_in_wheel = 0. RX: 1453K resps, 1453K reqs. Resps/batch: min 24K, max 24K. Latency: N/A. Rate = [25.00, 25.00, 25.00, 25.00 Gbps].

Did I miss anything? Thank you.

test failed in server_failure_test&multi_process_test

Hi, Anuj
My cluster has 4 mellanox connectx-4 nics: ib0 and ib1 are infiniband nics. p6p1 and p6p2 are ethernet nics.

  mlx5_0 port 1 ==> ib0 (Up)
  mlx5_1 port 1 ==> ib1 (Down)
  mlx5_2 port 1 ==> p6p1 (Up)
  mlx5_3 port 1 ==> p6p2 (Up)

Ofed version is :

  MLNX_OFED_LINUX-4.4-2.0.7.0

Operating system is:

 CentOS Linux release 7.5.1804

I have two questions:

When I run ctest and hello-world, from process output, we known, erpc automatically choose Device mlx5_3/p6p2,
What should I do to select another nic? (changing ip doesn't work.)

2.I compile with command "cmake . -DPERF=OFF -DTRANSPORT=raw", then run ctest.
However, server_failure_test and multi_process_test failed, the error info is:

Total Test time (real) =  56.93 sec

The following tests FAILED:
  8 - server_failure_test (OTHER_FAULT)
  9 - multi_process_test (OTHER_FAULT)
 Errors while running CTest

I run build/server_failure_test and build/multi_process_test, the error information is:

server_failure_test:

server_failure_test: /root/eRPC/tests/client_tests/server_failure_test.cc:93:
void generic_test_func(erpc::Nexus*, size_t): Assertion `c.num_rpc_resps == config_num_rpcs' failed.Aborted

multi_process_test:

6:070851 WARNG: Installing flow rule for Rpc 0. NUMA node = 0. Flow RX UDP port = 36454.
6:071238 WARNG: RawTransport created for Rpc ID 0. Device mlx5_3/p6p2, port 1. IPv4 
63.63.63.86, MAC ec:d:9a:c5:ba:bd. Datapath UDP port 36454.
......
multi_process_test: /root/eRPC/tests/client_tests/multi_process_test.cc:60: void 
process_proxy_thread_func(size_t, size_t): Assertion `c.num_rpc_resps == num_processes - 1' failed.
Aborted

Looking forward for your reply and thanks in advance.

Best regards
Thomas

Nexus segfault when there are offline CPUs

I got a segfault when running the create_session_test at the following line:

eRPC/src/nexus_impl/nexus.cc

Line 63 in dff4589

get_lcores_for_numa_node(numa_node).at(sm_thread_lcore_index));

The problem is that sm_thread_lcore_index is assigned to be the last lcore at line 61 without considering its status while get_lcores_for_numa_node returns only online lcores.

Compiling error with dpdk-19.11 on Ubuntu 18.04

Hi Anuj,

I got an error while compiling the eRPC on ubuntu 18.04.
Here is the log

# output from
~/eRPC$ cmake . -DPERF=OFF -DTRANSPORT=dpdk
-- The C compiler identification is GNU 7.4.0
-- The CXX compiler identification is GNU 7.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found GTest: /usr/lib/libgtest.a
-- Compilation not optimized for performance.
-- Profile-guided optimization is disabled.
-- Logging level = warn.
-- Debugging is enabled. Perf will be low.
-- Testing is enabled. Performance will be low.
-- LTO is disabled. Performance will be low.
-- Boost version: 1.65.1
-- Boost include directory = /usr/include
-- Selected transport = dpdk.
-- Compiling eRPC as a library
-- No autorun_app_file found. No application will be compiled.
-- Configuring done
-- Generating done
-- Build files have been written to: /home/ubuntu/eRPC

# Then errors
In file included from /home/ubuntu/eRPC/src/transport_impl/dpdk/dpdk_transport.cc:8:0:
/home/ubuntu/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static void erpc::DpdkTransport::install_flow_rule(size_t, size_t, uint32_t, uint16_t)’:
/home/ubuntu/eRPC/src/transport_impl/dpdk/dpdk_transport.h:125:69: error: ‘int rte_eth_dev_filter_supported(uint16_t, rte_filter_type)’ is deprecated [-Werror=deprecated-declarations]
     if (rte_eth_dev_filter_supported(phy_port, RTE_ETH_FILTER_NTUPLE) == 0) {
                                                                     ^
In file included from /home/ubuntu/eRPC/src/transport_impl/dpdk/dpdk_transport.h:17:0,
                 from /home/ubuntu/eRPC/src/transport_impl/dpdk/dpdk_transport.cc:8:
/usr/local/include/rte_ethdev.h:3553:5: note: declared here
 int rte_eth_dev_filter_supported(uint16_t port_id,
     ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from /home/ubuntu/eRPC/src/transport_impl/dpdk/dpdk_transport.cc:8:0:
/home/ubuntu/eRPC/src/transport_impl/dpdk/dpdk_transport.h:139:68: error: ‘int rte_eth_dev_filter_ctrl(uint16_t, rte_filter_type, rte_filter_op, void*)’ is deprecated [-Werror=deprecated-declarations]
                                         RTE_ETH_FILTER_ADD, &ntuple);
....
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:470: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_sm_api.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_sm_api.cc.o] Error 1
CMakeFiles/erpc.dir/build.make:182: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_rfr.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_rfr.cc.o] Error 1
CMakeFiles/erpc.dir/build.make:158: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_queues.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_queues.cc.o] Error 1
CMakeFiles/erpc.dir/build.make:374: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_rx.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_rx.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:302: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_ev_loop.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_ev_loop.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:398: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_connect_handlers.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_connect_handlers.cc.o] Error 1
CMakeFiles/Makefile2:363: recipe for target 'CMakeFiles/erpc.dir/all' failed
make[1]: *** [CMakeFiles/erpc.dir/all] Error 2
Makefile:94: recipe for target 'all' failed
make: *** [all] Error 2

Here is the full version of the log output: log

The system is running on AWS, I have installed dpdk library globally.
I see the first part of the errors are due to [-Werror=deprecated-declarations], is it safe to surpass error by removing the -Werror flag here?

eRPC not working over ethernet switch

Hi, I'm running the provided small_rpc_tput app on two machines. Both machines have ubuntu 18.04, kernel version "4.15.0" on "Intel(R) Xeon(R) Gold 5120 CPU @ 2.2" machines with 93 GB memory. Both machines have 56 cores and "MT27800 Family [ConnectX-5]" NICs running on latest version of Mellanox ofed drivers from their website. Port one of both NICs is connected directly and port 0 is connected via a 100GBE switch.

When I run the experiment on the direct link, it runs without any problem, but when I run it over the switch, I'm getting zero throughput.

From the output it seems switch is dropping all the packets and eRPC is trying to retransmit indefinitely.

ib_read_bw works fine over the switch. I also wrote a simple RDMA RoCE program that writes to remote memory and it works fine over the switch. I'm attaching the tcpdump output of interface when running small_rpc_tput.
erpcs.txt

Can you provide any insight as to why the switch might be dropping the packets?

Files scripts/autorun_app_file is not found

when run do.sh or other .sh files , File scripts/autorun_app_file is not found

How does this work?

In the red frame，some packages are received, but can it be guaranteed that this is a full package?
check_magic() is called in the blue frame, but how to guarantee that we have already received enough data for the struct pkthdr_t.
As far as I know, packages may be split during data transfer, so how can this works well?

Could eRPC raw model run without verbs

The raw model's code depends on verbs code. I want to know whether eRPC raw model could run without verbs ? Does raw UDP model have some special characteristics that depends on verbs to get good performance?

Build error with dpdk

Hi,
My dpdk version is dpdk-18.11.10 and Ubuntu 18.04.3.
I get error when run make.

n file included from /home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.cc:8:0:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.cc:8:0:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_queues.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_disconnect_handlers.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_disconnect_handlers.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_pkt_loss.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_req.cc:3:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport_datapath.cc:3:0:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc.cc:9:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_kick.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_resp.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_kick.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_ev_loop.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_queues.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_connect_handlers.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_resp.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_cr.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_rx.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_req.cc:3:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/apps/latency/../apps_common.h:9,
                 from /home/zty/eRPC/apps/latency/latency.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_reset_handlers.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_connect_handlers.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.cc: In member function ‘void erpc::DpdkTransport::setup_phy_port()’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.cc:126:36: error: ‘RTE_ETHER_MAX_LEN’ was not declared in this scope
   eth_conf.rxmode.max_rx_pkt_len = RTE_ETHER_MAX_LEN;
                                    ^~~~~~~~~~~~~~~~~
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.cc: In member function ‘void erpc::DpdkTransport::setup_phy_port()’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.cc:126:36: error: ‘RTE_ETHER_MAX_LEN’ was not declared in this scope
   eth_conf.rxmode.max_rx_pkt_len = RTE_ETHER_MAX_LEN;
                                    ^~~~~~~~~~~~~~~~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_sm_api.cc:8:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.cc:126:36: note: suggested alternative: ‘ETHER_MAX_LEN’
   eth_conf.rxmode.max_rx_pkt_len = RTE_ETHER_MAX_LEN;
                                    ^~~~~~~~~~~~~~~~~
                                    ETHER_MAX_LEN
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.cc: In member function ‘void erpc::DpdkTransport::resolve_phy_port()’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.cc:229:25: error: aggregate ‘erpc::DpdkTransport::resolve_phy_port()::rte_ether_addr ma’ has incomplete type and cannot be defined
   struct rte_ether_addr mac;
                         ^~~
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.cc:126:36: note: suggested alternative: ‘ETHER_MAX_LEN’
   eth_conf.rxmode.max_rx_pkt_len = RTE_ETHER_MAX_LEN;
                                    ^~~~~~~~~~~~~~~~~
                                    ETHER_MAX_LEN
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.cc: In member function ‘void erpc::DpdkTransport::resolve_phy_port()’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.cc:229:25: error: aggregate ‘erpc::DpdkTransport::resolve_phy_port()::rte_ether_addr ma’ has incomplete type and cannot be defined
   struct rte_ether_addr mac;
                         ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc.cc:9:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport_datapath.cc:3:0:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_sm_helpers.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_rx.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_pkt_loss.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_sm_helpers.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/session.h:8,
                 from /home/zty/eRPC/src/nexus.h:7,
                 from /home/zty/eRPC/src/nexus_impl/nexus_sm_thread.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/session.h:8,
                 from /home/zty/eRPC/src/nexus.h:7,
                 from /home/zty/eRPC/src/nexus_impl/nexus.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/session.h:8,
                 from /home/zty/eRPC/src/nexus.h:7,
                 from /home/zty/eRPC/src/nexus_impl/nexus.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/session.h:8,
                 from /home/zty/eRPC/src/nexus.h:7,
                 from /home/zty/eRPC/src/nexus_impl/nexus_sm_thread.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/session.h:8,
                 from /home/zty/eRPC/src/nexus.h:7,
                 from /home/zty/eRPC/src/nexus_impl/nexus_bg_thread.cc:2:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:566: recipe for target 'CMakeFiles/erpc.dir/src/transport_impl/dpdk/dpdk_transport_datapath.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/transport_impl/dpdk/dpdk_transport_datapath.cc.o] Error 1
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_rfr.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_fault_inject.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/latency.dir/build.make:590: recipe for target 'CMakeFiles/latency.dir/src/transport_impl/dpdk/dpdk_transport_datapath.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/transport_impl/dpdk/dpdk_transport_datapath.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/latency.dir/build.make:566: recipe for target 'CMakeFiles/latency.dir/src/transport_impl/dpdk/dpdk_transport.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/transport_impl/dpdk/dpdk_transport.cc.o] Error 1
CMakeFiles/erpc.dir/build.make:542: recipe for target 'CMakeFiles/erpc.dir/src/transport_impl/dpdk/dpdk_transport.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/transport_impl/dpdk/dpdk_transport.cc.o] Error 1
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_fault_inject.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/session.h:8,
                 from /home/zty/eRPC/src/nexus.h:7,
                 from /home/zty/eRPC/src/nexus_impl/nexus_bg_thread.cc:2:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_sm_api.cc:8:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_reset_handlers.cc:5:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_ev_loop.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:230: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_kick.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_kick.cc.o] Error 1
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_cr.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
In file included from /home/zty/eRPC/src/cc/timing_wheel.h:19:0,
                 from /home/zty/eRPC/src/rpc.h:4,
                 from /home/zty/eRPC/src/rpc_impl/rpc_rfr.cc:1:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h: In static member function ‘static uint32_t erpc::DpdkTransport::get_port_ipv4_addr(size_t)’:
/home/zty/eRPC/src/transport_impl/dpdk/dpdk_transport.h:92:27: error: aggregate ‘erpc::DpdkTransport::get_port_ipv4_addr(size_t)::rte_ether_addr mac’ has incomplete type and cannot be defined
     struct rte_ether_addr mac;
                           ^~~
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/latency.dir/build.make:446: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_disconnect_handlers.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_disconnect_handlers.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/latency.dir/build.make:182: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_queues.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_queues.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/latency.dir/build.make:110: recipe for target 'CMakeFiles/latency.dir/src/nexus_impl/nexus_bg_thread.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/nexus_impl/nexus_bg_thread.cc.o] Error 1
CMakeFiles/erpc.dir/build.make:110: recipe for target 'CMakeFiles/erpc.dir/src/nexus_impl/nexus_sm_thread.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/nexus_impl/nexus_sm_thread.cc.o] Error 1
CMakeFiles/latency.dir/build.make:134: recipe for target 'CMakeFiles/latency.dir/src/nexus_impl/nexus_sm_thread.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/nexus_impl/nexus_sm_thread.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:350: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_pkt_loss.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_pkt_loss.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: all warnings being treated as errors
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:206: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_cr.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_cr.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/latency.dir/build.make:398: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_rx.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_rx.cc.o] Error 1
CMakeFiles/erpc.dir/build.make:302: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_ev_loop.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_ev_loop.cc.o] Error 1
CMakeFiles/latency.dir/build.make:254: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_kick.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_kick.cc.o] Error 1
CMakeFiles/latency.dir/build.make:470: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_reset_handlers.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_reset_handlers.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/latency.dir/build.make:302: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_resp.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_resp.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:470: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_sm_api.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_sm_api.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:278: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_resp.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_resp.cc.o] Error 1
CMakeFiles/latency.dir/build.make:278: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_req.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_req.cc.o] Error 1
CMakeFiles/erpc.dir/build.make:158: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_queues.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_queues.cc.o] Error 1
CMakeFiles/erpc.dir/build.make:134: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:62: recipe for target 'CMakeFiles/erpc.dir/src/nexus_impl/nexus.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/nexus_impl/nexus.cc.o] Error 1
CMakeFiles/latency.dir/build.make:422: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_connect_handlers.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_connect_handlers.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:422: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_disconnect_handlers.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_disconnect_handlers.cc.o] Error 1
CMakeFiles/latency.dir/build.make:62: recipe for target 'CMakeFiles/latency.dir/apps/latency/latency.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/apps/latency/latency.cc.o] Error 1
CMakeFiles/erpc.dir/build.make:398: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_connect_handlers.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_connect_handlers.cc.o] Error 1
CMakeFiles/latency.dir/build.make:86: recipe for target 'CMakeFiles/latency.dir/src/nexus_impl/nexus.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/nexus_impl/nexus.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:254: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_req.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_req.cc.o] Error 1
CMakeFiles/erpc.dir/build.make:494: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_sm_helpers.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_sm_helpers.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:374: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_rx.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_rx.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/latency.dir/build.make:518: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_sm_helpers.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_sm_helpers.cc.o] Error 1
CMakeFiles/latency.dir/build.make:374: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_pkt_loss.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_pkt_loss.cc.o] Error 1
CMakeFiles/latency.dir/build.make:158: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:182: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_rfr.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_rfr.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:86: recipe for target 'CMakeFiles/erpc.dir/src/nexus_impl/nexus_bg_thread.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/nexus_impl/nexus_bg_thread.cc.o] Error 1
CMakeFiles/latency.dir/build.make:350: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_fault_inject.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_fault_inject.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:326: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_fault_inject.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_fault_inject.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/latency.dir/build.make:494: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_sm_api.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_sm_api.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/latency.dir/build.make:326: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_ev_loop.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_ev_loop.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/erpc.dir/build.make:446: recipe for target 'CMakeFiles/erpc.dir/src/rpc_impl/rpc_reset_handlers.cc.o' failed
make[2]: *** [CMakeFiles/erpc.dir/src/rpc_impl/rpc_reset_handlers.cc.o] Error 1
CMakeFiles/Makefile2:363: recipe for target 'CMakeFiles/erpc.dir/all' failed
make[1]: *** [CMakeFiles/erpc.dir/all] Error 2
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/latency.dir/build.make:230: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_cr.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_cr.cc.o] Error 1
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
CMakeFiles/latency.dir/build.make:206: recipe for target 'CMakeFiles/latency.dir/src/rpc_impl/rpc_rfr.cc.o' failed
make[2]: *** [CMakeFiles/latency.dir/src/rpc_impl/rpc_rfr.cc.o] Error 1
CMakeFiles/Makefile2:437: recipe for target 'CMakeFiles/latency.dir/all' failed
make[1]: *** [CMakeFiles/latency.dir/all] Error 2
Makefile:94: recipe for target 'all' failed
make: *** [all] Error 2

[Doc Request] Docs for erpc on AWS or Soft-RoCE

Hi,

Do you still plan to add docs about soft-roce or how to test it on aws cloud? Thanks!

Unremarkable bug for Mellanox ConnectX-6

ConnectX-6 is a new NIC with the higher speed (200Gb/s), while the port_attr.active_speed == 64. I think that the Compute the bandwidth in ./src/transport_impl/verbs_common.h:190 should be updated.

Using Modded Driver With CX5 (Eth)

Hi,

I'm running the latency app, and the latency seems rather high. The eRPC paper reports ~2.3us median latency with CX5 (Eth), but I'm getting ~5.3us:

$ ./scripts/do.sh 1 0
Installing modded drivers
do.sh: Launching process 1 on NUMA node 0
39:883042 WARNG: Modded driver unavailable. Performance will be low.
Process 1: Creating session to 10.0.1.98:31850.
Process 1: Session connected. Starting work.
write_size median_us 5th_us 99th_us 999th_us
64 5.3 5.1 6.4 9.2
128 5.3 5.1 6.4 9.2
256 5.4 5.2 6.5 9.4
512 5.5 5.3 6.7 9.4
1024 7.7 7.4 9.3 12.2
64 5.2 5.1 6.2 9.2
128 5.3 5.1 6.2 9.2
256 5.3 5.2 6.2 9.1
512 5.5 5.3 6.4 9.7
1024 7.7 7.4 9.0 12.8
64 5.2 5.1 6.2 9.5
128 5.3 5.1 6.2 9.7
256 5.3 5.2 6.2 9.8
512 5.5 5.3 6.4 9.8
1024 7.6 7.4 8.8 13.0
64 5.2 5.1 6.1 9.7
128 5.3 5.1 6.2 9.9
256 5.3 5.2 6.3 9.8
512 5.5 5.3 6.5 10.0
1024 7.6 7.4 8.9 13.6

I'm running this on two servers, each with 12 Intel E5-2603 v3 @ 1.60GHz and CX5 (Eth) NICs:

- MLNX_OFED_LINUX_VERSION: 4.7-1.0.0.1
- MLNX_OFED_ARCH: x86_64
- MLNX_OFED_DISTRO: ubuntu16.04
- distro: ubuntu16.04
- arch: x86_64
- kernel: 4.15.0-65-generic

This is how I configured and built eRPC:

echo latency > scripts/autorun_app_file
cmake . -DPERF=ON -DTRANSPORT=infiniband -DROCE=on -DLOG_LEVEL=warn
make -j12

I tried to use the modded driver. I built the driver with the following steps:

cd drivers/4.4/libmlx5-41mlnx1
./autogen.sh
./configure
make
./update-driver.sh

I see that this replaced /usr/lib/libmlx5.so.1.0.0 with the modded driver that was just built in drivers/4.4/libmlx5-41mlnx1/src/.libs/libmlx5.so.1.0.0.

However, when I try running eRPC again (the latency app), it warns that the modded driver is unavailable, and throws a fatal error:

$ ./scripts/do.sh 1 0
Installing modded drivers
do.sh: Launching process 0 on NUMA node 0
22:288858 WARNG: Modded driver unavailable. Performance will be low.
eRPC: Fatal error. Bad wc status 4231843.

Why doesn't eRPC detect the modded driver? Did I miss any steps? Do you have any suggestions for troubleshooting?

Thank you!

Question: You mentioned in README that eRPC will not perform well in AWS EC2 VM, why?

Can you please explain why eRPC not perform well in a KVM VM like AWS EC2? Any special features only support by Mellanox NIC depended by eRPC?

Is it possibe to enhance eRPC that can have a high performance in VM with a VIRT-IO ENI or a VF Pass-through vNic?

Thanks.

怎么样把DPDK源码集成到TARS里面？？？

主要目的是想用DPDK来接收海量的数据，但是先不处理。然后通过TARS来把这些数据进行处理。所以目前的问题就是如何把我们的DPDK和TARS融合在一起是个问题，求大牛们赐教？

When does transport layer make data visible to app, and in what granularity

Hi Anuj,

This issue maybe not eRPC specific, but more from RDMA's angle. I just thought you are the right person to ask.

Both eRPC and RDMA break big messages into small packets, and use go-back-N to handle packet loss. My concern here lies in: when does the transport layer make data visible to application, and in what granularity.

My understanding about eRPC: eRPC won't export the data to application until all packets of a message has been received. In another words, eRPC does not allow application to see a partial received message.

While RDMA (e.g., WRITE) is different: the RNIC will just write each packet directly into final DRAM location. That means if there is an application (e.g., FARM) polling DRAM, the application will see the new data the moment the first packet is received, and in packet granularity. One reason of this, is because RNIC does not have on-chip memory to hold out-of-order packets. The IRN paper sort of confirmed my understanding of this.

Do you think my understanding is correct? Thank you in advance.

Erpc Paper doesn’t compare against traditional udp

Maybe I’m overlooking some obvious element of the paper/system in the context of the existing literature and related systems, but it naively seems like it would be informative to compare against eg naive direct udp sockets api usage and or udp via something like tun/tap.

(I guess I’m generally interested in dist sys / network engineering where systems don’t need sudo privileges to run etc, and such. Maybe dpdk has stuff for that now since I last looked ? )

I’m not sure if a ticket is the right fora for this question but :)

Side nit: pfc is mentioned a lot in the paper, but you don’t provide a citation to any reference for those not in the know afaict

ibv related error when compile

I am trying to run the project and meet several problems.

I saw the Hello is move out. However, I still need to add the autorun_app_file with any app inorder to create the makefile form CMakeLink.txt.
I run the make -j after create the make with cmake . -DPERF=OFF -DTRANSPORT=infiniband. However, I met a set of ibv related error shown as following:

[ 2%] Building CXX object CMakeFiles/erpc.dir/src/transport_impl/raw/raw_transport.cc.o
[ 2%] Building CXX object CMakeFiles/erpc.dir/src/transport_impl/raw/raw_transport_datapath.cc.o
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport_datapath.cc: In member function ‘void erpc::RawTransport::post_recvs(size_t)’:
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport_datapath.cc:203:24: error: invalid use of incomplete type ‘struct erpc::ibv_exp_wq_family’
int ret = wq_family->recv_burst(wq, &mp_recv_sge[mp_sge_idx], 1);
^~
In file included from /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport_datapath.cc:1:0:
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.h:200:10: note: forward declaration of ‘struct erpc::ibv_exp_wq_family’
struct ibv_exp_wq_family wq_family;
^~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc: In destructor ‘erpc::RawTransport::~RawTransport()’:
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:53:40: error: aggregate ‘erpc::RawTransport::~RawTransport()::ibv_exp_release_intf_params rel_intf_params’ has incomplete type and cannot be defined
struct ibv_exp_release_intf_params rel_intf_params;
^~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:56:73: error: ‘ibv_exp_release_intf’ was not declared in this scope
ibv_exp_release_intf(resolve.ib_ctx, wq_family, &rel_intf_params) == 0,
^
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:65:54: error: ‘ibv_exp_destroy_rwq_ind_table’ was not declared in this scope
exit_assert(ibv_exp_destroy_rwq_ind_table(ind_tbl) == 0,
^
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:68:38: error: ‘ibv_exp_destroy_wq’ was not declared in this scope
exit_assert(ibv_exp_destroy_wq(wq) == 0, "Failed to destroy WQ");
^
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc: In member function ‘void erpc::RawTransport::init_mp_recv_qp()’:
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:260:31: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_wq_init_attr wq_init_attr’ has incomplete type and cannot be defined
struct ibv_exp_wq_init_attr wq_init_attr;
^~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:263:26: error: ‘IBV_EXP_WQT_RQ’ was not declared in this scope
wq_init_attr.wq_type = IBV_EXP_WQT_RQ;
^~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:269:29: error: ‘IBV_EXP_CREATE_WQ_MP_RQ’ was not declared in this scope
wq_init_attr.comp_mask |= IBV_EXP_CREATE_WQ_MP_RQ;
^~~~~~~~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:270:34: error: ‘IBV_EXP_MP_RQ_NO_SHIFT’ was not declared in this scope
wq_init_attr.mp_rq.use_shift = IBV_EXP_MP_RQ_NO_SHIFT;
^~~~~~~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:273:55: error: ‘ibv_exp_create_wq’ was not declared in this scope
wq = ibv_exp_create_wq(resolve.ib_ctx, &wq_init_attr);
^
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:277:26: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_wq_attr wq_attr’ has incomplete type and cannot be defined
struct ibv_exp_wq_attr wq_attr;
^~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:279:23: error: ‘IBV_EXP_WQ_ATTR_STATE’ was not declared in this scope
wq_attr.attr_mask = IBV_EXP_WQ_ATTR_STATE;
^~~~~~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:280:22: error: ‘IBV_EXP_WQS_RDY’ was not declared in this scope
wq_attr.wq_state = IBV_EXP_WQS_RDY;
^~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:281:43: error: ‘ibv_exp_modify_wq’ was not declared in this scope
rt_assert(ibv_exp_modify_wq(wq, &wq_attr) == 0, "Failed to ready WQ");
^
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:284:8: error: use of enum ‘ibv_exp_query_intf_status’ without previous declaration
enum ibv_exp_query_intf_status intf_status = IBV_EXP_INTF_STAT_OK;
^~~~~~~~~~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:284:48: error: ‘IBV_EXP_INTF_STAT_OK’ was not declared in this scope
enum ibv_exp_query_intf_status intf_status = IBV_EXP_INTF_STAT_OK;
^~~~~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:285:36: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_query_intf_params query_intf_params’ has incomplete type and cannot be defined
struct ibv_exp_query_intf_params query_intf_params;
^~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:287:34: error: ‘IBV_EXP_INTF_GLOBAL’ was not declared in this scope
query_intf_params.intf_scope = IBV_EXP_INTF_GLOBAL;
^~~~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:288:28: error: ‘IBV_EXP_INTF_WQ’ was not declared in this scope
query_intf_params.intf = IBV_EXP_INTF_WQ;
^~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:291:74: error: ‘ibv_exp_query_intf’ was not declared in this scope
ibv_exp_query_intf(resolve.ib_ctx, &query_intf_params, &intf_status));
^
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:295:42: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_rwq_ind_table_init_attr rwq_ind_table_init_attr’ has incomplete type and cannot be defined
struct ibv_exp_rwq_ind_table_init_attr rwq_ind_table_init_attr;
^~~~~~~~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:302:76: error: ‘ibv_exp_create_rwq_ind_table’ was not declared in this scope
ibv_exp_create_rwq_ind_table(resolve.ib_ctx, &rwq_ind_table_init_attr);
^
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:314:31: error: aggregate ‘erpc::RawTransport::init_mp_recv_qp()::ibv_exp_rx_hash_conf rx_hash_conf’ has incomplete type and cannot be defined
struct ibv_exp_rx_hash_conf rx_hash_conf;
^~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:316:35: error: ‘IBV_EXP_RX_HASH_FUNC_TOEPLITZ’ was not declared in this scope
rx_hash_conf.rx_hash_function = IBV_EXP_RX_HASH_FUNC_TOEPLITZ;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:319:38: error: ‘IBV_EXP_RX_HASH_DST_PORT_UDP’ was not declared in this scope
rx_hash_conf.rx_hash_fields_mask = IBV_EXP_RX_HASH_DST_PORT_UDP;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:326:28: error: ‘IBV_EXP_QP_INIT_ATTR_RX_HASH’ was not declared in this scope
IBV_EXP_QP_INIT_ATTR_RX_HASH;
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:327:16: error: ‘struct ibv_exp_qp_init_attr’ has no member named ‘rx_hash_conf’
qp_init_attr.rx_hash_conf = &rx_hash_conf;
^~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc: In member function ‘void erpc::RawTransport::install_flow_rule()’:
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:347:14: error: ‘ibv_exp_flow_spec_ipv4_ext’ was not declared in this scope
sizeof(ibv_exp_flow_spec_ipv4_ext) + sizeof(ibv_exp_flow_spec_tcp_udp);
^~~~~~~~~~~~~~~~~~~~~~~~~~
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:372:72: error: expected ‘>’ before ‘’ token
auto *spec_ipv4 = reinterpret_cast<struct ibv_exp_flow_spec_ipv4_ext >(buf);
^
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:372:72: error: expected ‘(’ before ‘’ token
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:372:73: error: expected primary-expression before ‘>’ token
auto *spec_ipv4 = reinterpret_cast<struct ibv_exp_flow_spec_ipv4_ext *>(buf);
^
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:372:79: error: expected ‘)’ before ‘;’ token
auto spec_ipv4 = reinterpret_cast<struct ibv_exp_flow_spec_ipv4_ext >(buf);
^
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:373:21: error: ‘IBV_EXP_FLOW_SPEC_IPV4_EXT’ was not declared in this scope
spec_ipv4->type = IBV_EXP_FLOW_SPEC_IPV4_EXT;
^~~~~~~~~~~~~~~~~~~~~~~~~~
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
make[2]: *** [CMakeFiles/erpc.dir/src/transport_impl/raw/raw_transport_datapath.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc: In member function ‘void erpc::RawTransport::init_recvs(uint8_t)’:
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:470:16: error: invalid use of incomplete type ‘struct erpc::ibv_exp_wq_family’
wq_family->recv_burst(wq, &mp_recv_sge[i], 1);
^~
In file included from /home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.cc:4:0:
/home/ql9/RDMA/eRPC/src/transport_impl/raw/raw_transport.h:200:10: note: forward declaration of ‘struct erpc::ibv_exp_wq_family’
struct ibv_exp_wq_family *wq_family;
^~~~~~~~~~~~~~~~~
At global scope:
cc1plus: error: unrecognized command line option ‘-Wno-keyword-macro’ [-Werror]
cc1plus: error: unrecognized command line option ‘-Wno-nested-anon-types’ [-Werror]
cc1plus: all warnings being treated as errors
make[2]: *** [CMakeFiles/erpc.dir/src/transport_impl/raw/raw_transport.cc.o] Error 1
make[1]: *** [CMakeFiles/erpc.dir/all] Error 2
make: *** [all] Error 2

I wonder what I did wrong during build and compile process. Thanks.

Question: In README you mentioned "eRPC not perform well in VM by design", why?

Can you please explain why eRPC not perform well in a KVM VM like AWS EC2? Any special features only support by Mellanox NIC that eRPC depends on? or any capability is missing in vNic of VM?

Is it possibe to enhance eRPC to performance well in VM with a VIRT-IO ENI or a VF Pass-through vNic?

Thanks.

The code is always using NIC port number 1.

I’m trying to build the provided apps with eRPC. I have three machines with dual port connect X5 Mellanox cards running Ubuntu 18.04. Following is the configuration of the machines.

Quads1:Port1(192.168.12.1) ---> Quads2:Port2(192.168.12.2);
Quads2:Port1(192.168.23.1) ---> Quads3:Port2(192.168.23.2);
Quads3:Port1(192.168.31.1) ---> Quads1:Port2(192.168.31.2);

I am trying to run the app “small_rpc_tput” on “quads1” and “quads2”. For some reason even though I provided IPs (192.168.12.1) and (192.168.12.2) in “scripts/autorun_process_file”, It uses port 1 (counting from 0) on both machines for connection. As you can see in the attached screenshot, on quads1 it should be using IP “192.168.12.1” but instead it is using “192.168.31.2”. The printed results also seem to imply that the connection is not established correctly.

Getting lower throughput than expected

Hi, I'm running the provided small_rpc_tput app, but I'm not getting the result that I expected based on the paper. Following is a screen shot for when I run it on two machines with 11 threads on each one. Batch size is 3 and concurrency is 60.

Each thread is serving about half million requests which is much lower than expected 5 million. Can you please share any insight as to why I'm getting such low throughput?

I'm running this experiment on ubuntu 18.04, kernel version "4.15.0" on "Intel(R) Xeon(R) Gold 5120 CPU @ 2.2" machines with 93 GB memory. Both machines have 56 cores and "MT27800 Family [ConnectX-5]" NICs running on latest version of Mellanox ofed drivers from their website.

failed to make with error: use of old-style cast

I met this error when I first ran on Boost 1.70.0.
Then I degraded to Boost 1.69.0. And I met the error again.
I noticed that there was an issue similar with my case for Boost 1.67+. So I degraded to Boost 1.61.0 and tried it again.
But the error still existed.
error output:
{CMAKE}/boost/function/function_template.hpp:638:69 error: use of old-style cast [-Werror=old-style-cast]
functor.member.obj_ref.obj_ptr = (void *)(f.get_pointer());

More output:
In file included from /home/wyl/.local/include/boost/function/detail/maybe_include.hpp:50:0,
from /home/wyl/.local/include/boost/function/detail/function_iterate.hpp:14,
from /home/wyl/.local/include/boost/preprocessor/iteration/detail/iter/forward1.hpp:72,
from /home/wyl/.local/include/boost/function.hpp:70,
from /home/wyl/.local/include/boost/algorithm/string/detail/find_iterator.hpp:18,
from /home/wyl/.local/include/boost/algorithm/string/find_iterator.hpp:24,
from /home/wyl/.local/include/boost/algorithm/string/iter_find.hpp:27,
from /home/wyl/.local/include/boost/algorithm/string/split.hpp:16,
from /home/wyl/.local/include/boost/algorithm/string.hpp:23,
from /home/wyl/eRPC/src/common.h:12,
from /home/wyl/eRPC/src/transport.h:8,
from /home/wyl/eRPC/src/transport_impl/infiniband/ib_transport.h:9,
from /home/wyl/eRPC/src/transport_impl/infiniband/ib_transport_datapath.cc:3:

erpc-io / erpc Goto Github PK

erpc's People

Contributors

Stargazers

Watchers

Forkers

erpc's Issues

Recommend Projects

Recommend Topics

Recommend Org