Dual port CMAC drops about open-nic HOT 3 OPEN

Hi @ample-alex,

The way you describe your test, makes me wonder what packet lengths that you are using and how bursty is the traffic when this is failing? (It makes me wonder whether somewhere in the design if FIFOs might be undersized and too shallow for coping with the amount of backpressure.) Does the drop rate increase with throughput beyond 5 Gb/s?

Also, can you give an example of how many processor cores you are assigning when you bind your dpdk driver? Are you assuming that each RX core can handle roughly ~5 Gb/s (or so)?
(Presumably though you would have enough because you were seeing 80+ Gbps being received in the single port case?)

Thanks,
--Chris

from open-nic.

Thanks Chris -

We're sending a mix of packets, with 50% of them being 1.5 KB, and the rest being an even mix between 60 and 1500 bytes. We do have a fair suspicion that the traffic is relatively bursty from our experiments with our single port design. The drop rate does increase with throughput after 5 Gbps.

We've tested with both 4 and 8 cores splitting the queues evenly between each core. 4 is enough for 80 Gbps on the single port, but the drops occur even with 8 cores on dual port. Are there FIFOs that you recommend we look at resizing?

Thanks again for the help.

from open-nic.

Without knowing what's going wrong, here are a couple of places within the design that I can suggest maybe probing first with Chipscope might give some insight to monitor if the FIFOs are filling, before increasing the FIFO sizes. Hopefully that could help to understand if these are the problem areas.

1 ) Completion FIFO in QDMA C2H Subsystem: maybe use Chipscope to check if that "cpl_fifo_full" occurs
2 ) At Packet Adapter RX: maybe use Chipscope to check if the "drop" signal gets asserted

Or if that's not possible, maybe just try experimentally increasing the FIFOs in these locations to see if there's any improvement.

Thanks,
--Chris

from open-nic.