Just with a simple go program <a href="https://github.com/CloudNativeDataPlane/cndp/fi

I found the problem, took me a while :-( <div class="snippet-clipboard-content not

receiving zero length packets,about cloudnativedataplane/cndp

Comments (3)

KeithWiles commented on July 28, 2024

I found the problem, took me a while :-(

"umems": {
     "umem0": {
         "bufcnt": 2,
         "bufsz": 2,
         "mtype": "1GB",
         "regions": [ 2 ],
         "rxdesc": 2,
         "txdesc": 2,
    },
},

It turns out the code looks correct, except AF_XDP rings have some restrictions. Here is the code from xskdev.c and comments.

    /*
     * We recommend that you set the fill ring size >= HW RX ring size +
     * AF_XDP RX ring size. Make sure you fill up the fill ring
     * with buffers at regular intervals, and you will with this setting
     * avoid allocation failures in the driver. These are usually quite
     * expensive since drivers have not been written to assume that
     * allocation failures are common. For regular sockets, kernel
     * allocated memory is used that only runs out in OOM situations
     * that should be rare.
     */
    umem_cfg.fill_size      = (cfg->rx_nb_desc * 2);
    umem_cfg.comp_size      = cfg->tx_nb_desc;
    umem_cfg.frame_size     = cfg->bufsz;
    umem_cfg.frame_headroom = cfg->buf_mgmt.buf_headroom;

    if (cfg->flags & LPORT_UMEM_UNALIGNED_BUFFERS)
        umem_cfg.flags = XDP_UMEM_UNALIGNED_CHUNK_FLAG;

    xu->fq_size = umem_cfg.fill_size;

    ret = netdev_get_ring_params(cfg->ifname, &hw_rx_nb_desc, NULL);
    if (ret)
        CNE_ERR("netdev_get_ring_params failure: %d\n", ret);
    else if (umem_cfg.fill_size < hw_rx_nb_desc + cfg->rx_nb_desc)
        CNE_INFO(
            "For %s recommend setting fill size (currently %d) to be >= HW RX ring size (%d) + "
            "AF_XDP Rx ring size (%d)\n",
            cfg->ifname, umem_cfg.fill_size, hw_rx_nb_desc, cfg->rx_nb_desc);

Note the FQ size must be >= (hw_rx_nb_desc + rx_nb_desc), but setting these values to does not work.
Please try the following values:

"umems": {
     "umem0": {
         "bufcnt": 8,
         "bufsz": 2,
         "mtype": "1GB",
         "regions": [ 8 ],
         "rxdesc": 2,
         "txdesc": 2,
    },
},

The minimum values appear to be this:

"umems": {
     "umem0": {
         "bufcnt": 4,
         "bufsz": 2,
         "mtype": "1GB",
         "regions": [ 4 ],
         "rxdesc": 1,
         "txdesc": 1,
    },
},

Will need to figure out the reason, but it appears the kernel AF_XDP code is failing and adding duplicate entries or pointing to freed buffers.

from cndp.

nwaples commented on July 28, 2024

Thanks.
What Ive found that works for the supplied example and the larger program we have is that I always need to have enough free mbuf entries to fill up the rx ring, fill ring and hw rx ring (which currently would be hw_rx_nb_desc + 3 * rx_nb_desc) to avoid any errors.
I had no errors as long as
bufcnt > hw_rx_nb_desc + 3 * rx_nb_desc + program caches/buffers

program caches/buffers = any program caches/buffers, such as the per thread mbuf cache (including the 1.5 cache multiplier),
the RxBurst receive slice in the example, lockess rings, etc.

Twiddling any of the above sizes I could get the errors, (even just not freeing the first N mbufs).

I can allocate the bufcnt with enough overhead. But I need to give it some margin just on the unlikely case the cache multiplier of fill ring size are changed in the future.
But because of this error I will also have to allocate a large enough bufcnt for the case where one lport could fill all the buffers in a multi threaded program (ddos etc).

from cndp.

KeithWiles commented on July 28, 2024

Resolved with a work around.

from cndp.

receiving zero length packets about cndp HOT 3 CLOSED

Comments (3)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent