imq / linuximq Goto Github PK

View Code? Open in Web Editor NEW

99.0 25.0 51.0 2.22 MB

Pseudo-driver for the intermediate queue device.

Home Page: https://imq.github.io/

License: GNU General Public License v2.0

Shell 100.00%

imq traffic-shaping bandwidth-saver traffic-control bandwidth-limiting virtual-interface qos rate-limiting

linuximq's People

Stargazers

Watchers

linuximq's Issues

fix two warnings

Hallo

I would kindly ask to include a two-line patch to address two warnings, mainly because they make compiling fail with -Werror.
The warnings are about the skb_copy_stored_cb function which is defined but not used (it's ony use is commented to date), and a pointer cast to an integer.

Thank you for your work!!

There it is:

--- a/net/core/skbuff.c.orig    2015-08-12 07:54:58.855459871 +0200
+++ b/net/core/skbuff.c 2015-08-12 08:04:12.678420885 +0200
@@ -138,6 +138,7 @@
 }
 EXPORT_SYMBOL(skb_restore_cb);

+static void skb_copy_stored_cb(struct sk_buff *   , const struct sk_buff *     ) __attribute__ ((unused));
 static void skb_copy_stored_cb(struct sk_buff *new, const struct sk_buff *__old)
 {
    struct skb_cb_table *next;
@@ -779,7 +780,7 @@
    while (skb->cb_next != NULL) {
        if (net_ratelimit())
            pr_warn("IMQ: kfree_skb: skb->cb_next: %08x\n",
-               (unsigned int)skb->cb_next);
+               (unsigned int)(uintptr_t)skb->cb_next);

        skb_restore_cb(skb);
    }

The patch for kernel 4.9 is incomplete

Fix is available in the pull-request #57

Patch fails with linux-3.13.10-hardened (Gentoo Linux)

Patch fails as shown below:

GENTOO linux-3.13.10-hardened # patch -p1 --dry-run < patch-imqmq-3.13.diff 
patching file drivers/net/Kconfig
patching file drivers/net/Makefile
patching file drivers/net/imq.c
patching file include/linux/imq.h
patching file include/linux/netfilter/xt_IMQ.h
patching file include/linux/netfilter_ipv4/ipt_IMQ.h
patching file include/linux/netfilter_ipv6/ip6t_IMQ.h
patching file include/linux/skbuff.h
Hunk #6 succeeded at 2656 (offset 4 lines).
patching file include/net/netfilter/nf_queue.h
patching file include/uapi/linux/netfilter.h
patching file net/core/dev.c
Hunk #2 succeeded at 2606 (offset 2 lines).
patching file net/core/skbuff.c
Hunk #1 succeeded at 75 with fuzz 2 (offset 2 lines).
Hunk #2 succeeded at 3421 with fuzz 2 (offset 3326 lines).
Hunk #3 FAILED at 668.
Hunk #4 FAILED at 800.
Hunk #5 FAILED at 3234.
3 out of 5 hunks FAILED -- saving rejects to file net/core/skbuff.c.rej
patching file net/ipv6/ip6_output.c
patching file net/netfilter/Kconfig
patching file net/netfilter/Makefile
patching file net/netfilter/core.c
patching file net/netfilter/nf_internals.h
patching file net/netfilter/nf_queue.c
patching file net/netfilter/xt_IMQ.c

Hardened sources is a security enhanced Linux kernel. Patches are available from this location: http://dev.gentoo.org/~blueness/hardened-sources/hardened-patches/hardened-patches-3.13.10-1.extras.tar.bz2

Error while comping kernel 4.4.33

While compiling i get this error:


bash-4.3# make -j1 INSTALL_MOD_STRIP=1 bzImage 
  CHK     include/config/kernel.release
  CHK     include/generated/uapi/linux/version.h
  CHK     include/generated/utsrelease.h
  CHK     include/generated/bounds.h
  CHK     include/generated/timeconst.h
  CHK     include/generated/asm-offsets.h
  CALL    scripts/checksyscalls.sh
  CHK     include/generated/compile.h
  CHK     kernel/config_data.h
  CC      net/core/dev.o
In file included from include/linux/linkage.h:6:0,
                 from include/linux/kernel.h:6,
                 from include/asm-generic/bug.h:13,
                 from ./arch/x86/include/asm/bug.h:35,
                 from include/linux/bug.h:4,
                 from include/linux/thread_info.h:11,
                 from ./arch/x86/include/asm/uaccess.h:8,
                 from net/core/dev.c:75:
include/linux/export.h:59:20: error: redefinition of ‘__kstrtab_validate_xmit_skb_list’
  static const char __kstrtab_##sym[]   \
                    ^
include/linux/export.h:72:2: note: in expansion of macro ‘__EXPORT_SYMBOL’
  __EXPORT_SYMBOL(sym, "_gpl")
  ^
net/core/dev.c:2849:1: note: in expansion of macro ‘EXPORT_SYMBOL_GPL’
 EXPORT_SYMBOL_GPL(validate_xmit_skb_list);
 ^
include/linux/export.h:59:20: note: previous definition of ‘__kstrtab_validate_xmit_skb_list’ was here
  static const char __kstrtab_##sym[]   \
                    ^
include/linux/export.h:69:2: note: in expansion of macro ‘__EXPORT_SYMBOL’
  __EXPORT_SYMBOL(sym, "")
  ^
net/core/dev.c:2821:1: note: in expansion of macro ‘EXPORT_SYMBOL’
 EXPORT_SYMBOL(validate_xmit_skb_list);
 ^
include/linux/export.h:63:39: error: redefinition of ‘__ksymtab_validate_xmit_skb_list’
  __visible const struct kernel_symbol __ksymtab_##sym \
                                       ^
include/linux/export.h:72:2: note: in expansion of macro ‘__EXPORT_SYMBOL’
  __EXPORT_SYMBOL(sym, "_gpl")
  ^
net/core/dev.c:2849:1: note: in expansion of macro ‘EXPORT_SYMBOL_GPL’
 EXPORT_SYMBOL_GPL(validate_xmit_skb_list);
 ^
include/linux/export.h:63:39: note: previous definition of ‘__ksymtab_validate_xmit_skb_list’ was here
  __visible const struct kernel_symbol __ksymtab_##sym \
                                       ^
include/linux/export.h:69:2: note: in expansion of macro ‘__EXPORT_SYMBOL’
  __EXPORT_SYMBOL(sym, "")
  ^
net/core/dev.c:2821:1: note: in expansion of macro ‘EXPORT_SYMBOL’
 EXPORT_SYMBOL(validate_xmit_skb_list);
 ^
make[2]: *** [net/core/dev.o] Error 1
make[1]: *** [net/core] Error 2
make: *** [net] Error 2

I think it still compiled with version 4.4.30.

I'm using this imq patch: linux-4.4.5-imq.diff

New patch for kernel 5.0 not work

Hi
i try to build with new kernel and imq but have problem:

CALL scripts/checksyscalls.sh
CC [M] drivers/net/imq.o
CC [M] drivers/net/tun.o
CC [M] drivers/net/nlmon.o
CC [M] drivers/net/sungem_phy.o
drivers/net/imq.c: In function 'nf_bridge_adjust_skb_data':
drivers/net/imq.c:409:11: error: 'struct sk_buff' has no member named 'nf_bridge'; did you mean 'nf_trace'?
if (skb->nf_bridge)
^~~~~~~~~
nf_trace
drivers/net/imq.c: In function 'nf_bridge_adjust_segmented_data':
drivers/net/imq.c:415:11: error: 'struct sk_buff' has no member named 'nf_bridge'; did you mean 'nf_trace'?
if (skb->nf_bridge)
^~~~~~~~~
nf_trace

Patch from Kernel 5.1 will not work with 5.2

I try to build a new kernel 5.2.0 (x86 and X86_64) including linuximq.
Here i use the patch from kernel 5.1, but the build will fail.
The fault will be:

In file included from ./include/linux/filter.h:25:0,
from arch/x86/net/bpf_jit_comp.c:9:
./include/net/sch_generic.h: In function 'qdisc_enqueue_root':
./include/net/sch_generic.h:149:5: error: implicit declaration of function 'qdisc_skb_cb' [-Werror=implicit-function-declaration]
qdisc_skb_cb(skb)->pkt_len = skb->len;
^~~~~~~~~~~~
./include/net/sch_generic.h:149:22: error: invalid type argument of '->' (have 'int')
qdisc_skb_cb(skb)->pkt_len = skb->len;
^~
./include/net/sch_generic.h:150:12: error: implicit declaration of function 'qdisc_enqueue'; did you mean 'qdisc_enqueue_root'? [-Werror=implicit-function-declaration]
return qdisc_enqueue(skb, sch, to_free) & NET_XMIT_MASK;
^~~~~~~~~~~~~
qdisc_enqueue_root
./include/net/sch_generic.h: At top level:
./include/net/sch_generic.h:515:36: error: conflicting types for 'qdisc_skb_cb'
static inline struct qdisc_skb_cb *qdisc_skb_cb(const struct sk_buff *skb)
^~~~~~~~~~~~
./include/net/sch_generic.h:149:5: note: previous implicit declaration of 'qdisc_skb_cb' was here
qdisc_skb_cb(skb)->pkt_len = skb->len;
^~~~~~~~~~~~
./include/net/sch_generic.h:827:19: error: static declaration of 'qdisc_enqueue' follows non-static declaration
static inline int qdisc_enqueue(struct sk_buff *skb, struct Qdisc *sch,
^~~~~~~~~~~~~
./include/net/sch_generic.h:150:12: note: previous implicit declaration of 'qdisc_enqueue' was here
return qdisc_enqueue(skb, sch, to_free) & NET_XMIT_MASK;
^~~~~~~~~~~~~

Kernel 4.4.X crash on IMQ

Hello,

we use IMQ for long time since kernel 2.6.X for shaping half duplex radio links between our linux routers. We do that using iptables mangle table and hooking PREROUTING input and POSTROUTING output via same physical ethernet device into IMQ device and than attaching complex HTB qdisc into it. It works OK on 2.6 and 3.4 kernels, unfortunately we getting crash on every single 4.4 version we tried. (4.4.26, 4.4.32, 4.4.38) Under real traffic such crash is almost immediate (please see traces for 4.4.32 and 4.4.38)

Was there any change for IMQ which prevents such usage, or is it some IMQ/kernel bug? Thank you for any help.

frog# [  232.395366] NMI watchdog: BUG: soft lockup - CPU#0 stuck for 22s! [swap per/0:0]
[  232.398366] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [swapper/1: 0]
[  232.398445] Modules linked in: xt_recent xt_nat xt_set ip_set_bitmap_ipmac ip _set_hash_ip ip_set nfnetlink xt_IMQ cls_u32 sch_sfq sch_htb imq xt_ACCOUNT(O) i pt_REJECT nf_reject_ipv4 xt_tcpudp xt_comment xt_hashlimit xt_multiport xt_addrt ype xt_conntrack xt_mark iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ ipv4 nf_nat_ipv4 iptable_raw ip6table_filter ip6_tables iptable_filter ip_tables x_tables ipv6 acpi_cpufreq processor i2c_piix4 uas sp5100_tco k10temp r8169 mii i2c_dev i2c_core nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_conntrack_b roadcast nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_f tp nf_nat_amanda nf_nat nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_ conntrack_proto_gre nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp ts_kmp n f_conntrack_amanda nf_conntrack ixgbe(O) hwmon dca vxlan udp_tunnel ip6_udp_tunn el e1000e(O) ptp pps_core e1000
[  232.398467] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O 4.4.32-x8 6_64 #1
[  232.398469] Hardware name: PC Engines APU/APU, BIOS 4.0 09/08/2014
[  232.398474] task: ffff88007b0ce180 ti: ffff88007b15c000 task.ti: ffff88007b15 c000
[  232.398488] RIP: 0010:[<ffffffff8108891e>] [<ffffffff8108891e>] queued_spin_ lock_slowpath+0x13e/0x150
[  232.398491] RSP: 0018:ffff88007dd038b8  EFLAGS: 00000202
[  232.398494] RAX: 0000000000000101 RBX: ffff880079693e00 RCX: 0000000000000001
[  232.398496] RDX: 0000000000000101 RSI: 0000000000000001 RDI: ffff880079693e80
[  232.398498] RBP: 0000000000000001 R08: 0000000000000101 R09: 00000000000000af
[  232.398501] R10: 0000000000000010 R11: 0000000000000004 R12: 0000000000000002
[  232.398503] R13: ffff88006d0b6400 R14: 0000000000000000 R15: ffff880079264000
[  232.398507] FS:  00007fd6660106a0(0000) GS:ffff88007dd00000(0000) knlGS:00000 00000000000
[  232.398510] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  232.398512] CR2: 00007f441d884a8f CR3: 000000007806f000 CR4: 00000000000006e0
[  232.398514] Stack:
[  232.398520]  ffffffffa02e7451 ffff88006d03b900 0000000100000001 ffff88007b0c6 000
[  232.398525]  01ffc900023ed010 0000000000000004 60a7eb034486e16d ffff88006d0b6 400
[  232.398530]  0000000000000000 ffff88006d0b6400 ffff880079264000 ffff88006d03b 900
[  232.398531] Call Trace:
[  232.398546]  <IRQ>
[  232.398546]  [<ffffffffa02e7451>] ? __imq_nf_queue+0x231/0x6f0 [imq]
[  232.398554]  [<ffffffffa02e7b42>] ? imq_nf_queue+0x232/0x376 [imq]
[  232.398563]  [<ffffffff814991c0>] ? ip_fragment.constprop.5+0x80/0x80
[  232.398568]  [<ffffffff8148e020>] ? nf_queue+0x130/0x190
[  232.398574]  [<ffffffff8148d066>] ? nf_hook_slow+0x56/0xd0
[  232.398579]  [<ffffffff81499d74>] ? ip_output+0xd4/0xf0
[  232.398584]  [<ffffffff814991c0>] ? ip_fragment.constprop.5+0x80/0x80
[  232.398589]  [<ffffffff81495dfa>] ? ip_forward+0x39a/0x490
[  232.398594]  [<ffffffff814959f0>] ? ip_frag_mem+0x40/0x40
[  232.398599]  [<ffffffff8148e157>] ? nf_reinject+0xd7/0x160
[  232.398604]  [<ffffffffa02e71a9>] ? imq_dev_xmit+0x89/0xc0 [imq]
[  232.398611]  [<ffffffff8145aa77>] ? dev_hard_start_xmit+0xa7/0x3e0
[  232.398617]  [<ffffffffa02e7408>] ? __imq_nf_queue+0x1e8/0x6f0 [imq]
[  232.398624]  [<ffffffffa02e7b42>] ? imq_nf_queue+0x232/0x376 [imq]
[  232.398630]  [<ffffffff81493d80>] ? ip_local_deliver_finish+0x1d0/0x1d0
[  232.398634]  [<ffffffff8148e020>] ? nf_queue+0x130/0x190
[  232.398639]  [<ffffffff8148d066>] ? nf_hook_slow+0x56/0xd0
[  232.398643]  [<ffffffff814945d6>] ? ip_rcv+0x306/0x3a0
[  232.398649]  [<ffffffff81493d80>] ? ip_local_deliver_finish+0x1d0/0x1d0
[  232.398654]  [<ffffffff814566e9>] ? __netif_receive_skb_core+0x6d9/0xa20
[  232.398659]  [<ffffffff814977b6>] ? ip_finish_output2+0x1c6/0x330
[  232.398666]  [<ffffffff814cd48a>] ? inet_gro_receive+0x1fa/0x240
[  232.398671]  [<ffffffff81458e3f>] ? netif_receive_skb_internal+0x2f/0xa0
[  232.398676]  [<ffffffff81459712>] ? napi_gro_receive+0xb2/0x100
[  232.398685]  [<ffffffffa018424d>] ? rtl8169_poll+0x2cd/0x600 [r8169]
[  232.398691]  [<ffffffff8145a784>] ? net_rx_action+0x204/0x340
[  232.398697]  [<ffffffff81054f4a>] ? __do_softirq+0xfa/0x290
[  232.398702]  [<ffffffff81055215>] ? irq_exit+0x95/0xa0
[  232.398707]  [<ffffffff8150e0bf>] ? do_IRQ+0x4f/0xd0
[  232.398713]  [<ffffffff8150c63f>] ? common_interrupt+0x7f/0x7f
[  232.398720]  <EOI>
[  232.398720]  [<ffffffff81087281>] ? cpu_startup_entry+0x111/0x330
[  232.398726]  [<ffffffff81036017>] ? start_secondary+0x137/0x160
[  232.398779] Code: 80 c6 07 01 48 8b 02 48 85 c0 75 0a f3 90 48 8b 02 48 85 c0 74 f6 c7 40 08 01 00 00 00 e9 5f ff ff ff 83 fa 01 75 04 eb 11 f3 90 <8b> 07 84 c0 75 f8 b8 01 00 00 00 66 89 07 c3 f3 c3 90 0f 1f 44
[  232.820313] Modules linked in: xt_recent xt_nat xt_set ip_set_bitmap_ipmac ip _set_hash_ip ip_set nfnetlink xt_IMQ cls_u32 sch_sfq sch_htb imq xt_ACCOUNT(O) i pt_REJECT nf_reject_ipv4 xt_tcpudp xt_comment xt_hashlimit xt_multiport xt_addrt ype xt_conntrack xt_mark iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ ipv4 nf_nat_ipv4 iptable_raw ip6table_filter ip6_tables iptable_filter ip_tables x_tables ipv6 acpi_cpufreq processor i2c_piix4 uas sp5100_tco k10temp r8169 mii i2c_dev i2c_core nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_conntrack_b roadcast nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_f tp nf_nat_amanda nf_nat nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_ conntrack_proto_gre nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp ts_kmp n f_conntrack_amanda nf_conntrack ixgbe(O) hwmon dca vxlan udp_tunnel ip6_udp_tunn el e1000e(O) ptp pps_core e1000
[  232.901484] CPU: 0 PID: 0 Comm: swapper/0 Tainted: G O L 4.4.32-x8 6_64 #1
[  232.909228] Hardware name: PC Engines APU/APU, BIOS 4.0 09/08/2014
[  232.915410] task: ffffffff8180f4c0 ti: ffffffff81800000 task.ti: ffffffff8180 0000
[  232.922897] RIP: 0010:[<ffffffff81088920>] [<ffffffff81088920>] queued_spin_ lock_slowpath+0x140/0x150
[  232.932232] RSP: 0018:ffff88007dc038a0  EFLAGS: 00000202
[  232.937545] RAX: 0000000000000101 RBX: ffff880079692c00 RCX: 0000000000000001
[  232.944677] RDX: 0000000000000101 RSI: 0000000000000001 RDI: ffff880079692c80
[  232.951811] RBP: 0000000000000000 R08: 0000000000000101 R09: 00000000000000af
[  232.958945] R10: 0000000000000010 R11: 0000000000000004 R12: 0000000000000002
[  232.966078] R13: ffff88007837f900 R14: 0000000000000000 R15: ffff880079022000
[  232.973213] FS:  00007f349ab196a0(0000) GS:ffff88007dc00000(0000) knlGS:00000 00000000000
[  232.981308] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  232.987053] CR2: 0000000000685318 CR3: 000000007807a000 CR4: 00000000000006f0
[  232.994186] Stack:
[  232.996206]  ffffffffa02e7451 ffff880078052100 0000000100000001 ffff88007b0c4 000
[  233.003703]  01ffc900023ed010 0000000000000004 792c55f3324573ba ffff88007837f 900
[  233.011200]  0000000000000003 ffff88007837f900 ffff880079022000 ffff880078052 100
[  233.018696] Call Trace:
[  233.021150]  <IRQ>
[  233.023093]  [<ffffffffa02e7451>] ? __imq_nf_queue+0x231/0x6f0 [imq]
[  233.029649]  [<ffffffffa02e7b42>] ? imq_nf_queue+0x232/0x376 [imq]
[  233.035841]  [<ffffffff814991c0>] ? ip_fragment.constprop.5+0x80/0x80
[  233.042284]  [<ffffffff8148e020>] ? nf_queue+0x130/0x190
[  233.047605]  [<ffffffff8148d066>] ? nf_hook_slow+0x56/0xd0
[  233.053092]  [<ffffffff81499d74>] ? ip_output+0xd4/0xf0
[  233.058328]  [<ffffffff814991c0>] ? ip_fragment.constprop.5+0x80/0x80
[  233.064775]  [<ffffffff81495dfa>] ? ip_forward+0x39a/0x490
[  233.070263]  [<ffffffff814959f0>] ? ip_frag_mem+0x40/0x40
[  233.075671]  [<ffffffff8148e157>] ? nf_reinject+0xd7/0x160
[  233.081168]  [<ffffffffa02e71a9>] ? imq_dev_xmit+0x89/0xc0 [imq]
[  233.087184]  [<ffffffff8145aa77>] ? dev_hard_start_xmit+0xa7/0x3e0
[  233.093372]  [<ffffffffa02e7408>] ? __imq_nf_queue+0x1e8/0x6f0 [imq]
[  233.099732]  [<ffffffffa02e7b42>] ? imq_nf_queue+0x232/0x376 [imq]
[  233.105920]  [<ffffffff81493d80>] ? ip_local_deliver_finish+0x1d0/0x1d0
[  233.112542]  [<ffffffff8148e020>] ? nf_queue+0x130/0x190
[  233.117863]  [<ffffffff8148d066>] ? nf_hook_slow+0x56/0xd0
[  233.123358]  [<ffffffff814945d6>] ? ip_rcv+0x306/0x3a0
[  233.128508]  [<ffffffff81493d80>] ? ip_local_deliver_finish+0x1d0/0x1d0
[  233.135129]  [<ffffffff814566e9>] ? __netif_receive_skb_core+0x6d9/0xa20
[  233.141837]  [<ffffffff814566e9>] ? __netif_receive_skb_core+0x6d9/0xa20
[  233.148543]  [<ffffffff810a4011>] ? ktime_get+0x31/0xa0
[  233.153772]  [<ffffffff81458e3f>] ? netif_receive_skb_internal+0x2f/0xa0
[  233.160480]  [<ffffffff81459015>] ? napi_gro_flush+0x55/0x70
[  233.166141]  [<ffffffff81459091>] ? napi_complete_done+0x61/0x90
[  233.172167]  [<ffffffffa0184017>] ? rtl8169_poll+0x97/0x600 [r8169]
[  233.178441]  [<ffffffff8145a784>] ? net_rx_action+0x204/0x340
[  233.184197]  [<ffffffff81054f4a>] ? __do_softirq+0xfa/0x290
[  233.189776]  [<ffffffff81055215>] ? irq_exit+0x95/0xa0
[  233.194925]  [<ffffffff8150e0bf>] ? do_IRQ+0x4f/0xd0
[  233.199905]  [<ffffffff8150c63f>] ? common_interrupt+0x7f/0x7f
[  233.205737]  <EOI>
[  233.207677]  [<ffffffff81087275>] ? cpu_startup_entry+0x105/0x330
[  233.213976]  [<ffffffff8108724e>] ? cpu_startup_entry+0xde/0x330
[  233.219992]  [<ffffffff818c4ed9>] ? start_kernel+0x406/0x426
[  233.225656] Code: 07 01 48 8b 02 48 85 c0 75 0a f3 90 48 8b 02 48 85 c0 74 f6 c7 40 08 01 00 00 00 e9 5f ff ff ff 83 fa 01 75 04 eb 11 f3 90 8b 07 <84> c0 75 f8 b8 01 00 00 00 66 89 07 c3 f3 c3 90 0f 1f 44 00 00

[  400.123003]  01ffc90002db5010 0000000000000004 dabab543e5e0a561 ffff88002fcf8d00
[  400.123003]  0000000000000002 ffff88002fcf8d00 ffff88003d3a5800 ffff88003ab97e00
[  400.123003] Call Trace:
[  400.123003]  <IRQ>
[  400.123003]  [<ffffffffa02c3451>] ? __imq_nf_queue+0x231/0x6f0 [imq]
[  400.123003]  [<ffffffffa02c3b42>] ? imq_nf_queue+0x232/0x376 [imq]
[  400.123003]  [<ffffffff814995a0>] ? ip_fragment.constprop.5+0x80/0x80
[  400.123003]  [<ffffffff8148e330>] ? nf_queue+0x130/0x190
[  400.123003]  [<ffffffff8148d396>] ? nf_hook_slow+0x56/0xd0
[  400.123003]  [<ffffffff8149a164>] ? ip_output+0xd4/0xf0
[  400.123003]  [<ffffffff814995a0>] ? ip_fragment.constprop.5+0x80/0x80
[  400.123003]  [<ffffffff814961da>] ? ip_forward+0x39a/0x490
[  400.123003]  [<ffffffff81495dd0>] ? ip_frag_mem+0x40/0x40
[  400.123003]  [<ffffffff8148e467>] ? nf_reinject+0xd7/0x160
[  400.123003]  [<ffffffffa02c31a9>] ? imq_dev_xmit+0x89/0xc0 [imq]
[  400.123003]  [<ffffffff8145ad17>] ? dev_hard_start_xmit+0xa7/0x3e0
[  400.123003]  [<ffffffffa02c3408>] ? __imq_nf_queue+0x1e8/0x6f0 [imq]
[  400.123003]  [<ffffffffa02c3b42>] ? imq_nf_queue+0x232/0x376 [imq]
[  400.123003]  [<ffffffff81494160>] ? ip_local_deliver_finish+0x1d0/0x1d0
[  400.123003]  [<ffffffff8148e330>] ? nf_queue+0x130/0x190
[  400.123003]  [<ffffffff8148d396>] ? nf_hook_slow+0x56/0xd0
[  400.123003]  [<ffffffff814949b6>] ? ip_rcv+0x306/0x3a0
[  400.123003]  [<ffffffff81494160>] ? ip_local_deliver_finish+0x1d0/0x1d0
[  400.123003]  [<ffffffff81456989>] ? __netif_receive_skb_core+0x6d9/0xa20
[  400.123003]  [<ffffffff814590df>] ? netif_receive_skb_internal+0x2f/0xa0
[  400.123003]  [<ffffffff814592b5>] ? napi_gro_flush+0x55/0x70
[  400.123003]  [<ffffffff81459331>] ? napi_complete_done+0x61/0x90
[  400.123003]  [<ffffffffa003724f>] ? e1000e_write_itr+0x154f/0x1740 [e1000e]
[  400.123003]  [<ffffffff8145aa24>] ? net_rx_action+0x204/0x340
[  400.123003]  [<ffffffff81054f3a>] ? __do_softirq+0xfa/0x290
[  400.123003]  [<ffffffff81055205>] ? irq_exit+0x95/0xa0
[  400.123003]  [<ffffffff8150e53f>] ? do_IRQ+0x4f/0xd0
[  400.123003]  [<ffffffff8150cabf>] ? common_interrupt+0x7f/0x7f
[  400.123003]  <EOI>
[  400.123003]  [<ffffffff81087271>] ? cpu_startup_entry+0x111/0x330
[  400.123003]  [<ffffffff81036007>] ? start_secondary+0x137/0x160
[  400.123003] Code: c1 74 80 c6 07 01 48 8b 02 48 85 c0 75 0a f3 90 48 8b 02 48 85 c0 74 f6 c7 40 08 01 00 00 00 e9 5f ff ff ff 83 fa 01 75 04 eb 11 <f3> 90 8b 07 84 c0 75 f8 b8 01 00 00 00 66 89 07 c3 f3 c3 90 66
[  415.085003] INFO: rcu_sched detected stalls on CPUs/tasks:
[  415.085003]  0-...: (12 GPs behind) idle=a1f/1/0 softirq=92466/92466 fqs=0
[  415.085003]  (detected by 1, t=240007 jiffies, g=33893, c=33892, q=3)
[  415.085003] Task dump for CPU 0:
[  415.085003] swapper/0       R  running task        0     0 0 0x00200008
[  415.085003]  0000000000200000 0100000000000000 ffffffff8182d1d8 ffff88003fa0c8e0
[  415.085003]  0100000000000000 ffffffffffffff1e ffffffff8108726d 0000000000000010
[  415.085003]  0000000000000202 ffffffff81803f68 0000000000000018 ffffffff8108723e
[  415.085003] Call Trace:
[  415.085003]  [<ffffffff8108726d>] ? cpu_startup_entry+0x10d/0x330
[  415.085003]  [<ffffffff8108723e>] ? cpu_startup_entry+0xde/0x330
[  415.085003]  [<ffffffff818c4ed9>] ? start_kernel+0x406/0x426
[  415.085003] rcu_sched kthread starved for 240007 jiffies! g33893 c33892 f0x2 s3 ->state=0x0
[  440.123003] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [swapper/1:0]
[  440.123003] Modules linked in: xt_recent xt_nat xt_set ip_set_bitmap_ipmac ip_set_hash_ip ip_set nfnetlink xt_IMQ cls_u32 sch_sfq sch_htb imq xt_ACCOUNT(O) ipt_REJECT nf_reject_ipv4 xt_tcpudp xt_comment xt_hashlimit xt_multiport xt_addrtype xt_conntrack xt_mark iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 iptable_raw ip6table_filter ip6_tables iptable_filter ip_tables x_tables 8021q mrp garp stp llc ipv6 parport_pc parport k8temp powernow_k8 processor i2c_nforce2 i2c_dev i2c_core nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_conntrack_broadcast nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda nf_nat nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp ts_kmp nf_conntrack_amanda nf_conntrack ixgbe(O) hwmon dca vxlan udp_tunnel ip6_udp_tunnel e1000e(O) ptp pps_core e1000
[  440.123003] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O L  4.4.38-x86_64 #2
[  440.123003] Hardware name: System manufacturer System Product Name/M2N-XE, BIOS 0107    12/10/2007
[  440.123003] task: ffff88003e0f6180 ti: ffff88003e174000 task.ti: ffff88003e174000
[  440.123003] RIP: 0010:[<ffffffff8108890c>] [<ffffffff8108890c>] queued_spin_lock_slowpath+0x13c/0x150
[  440.123003] RSP: 0018:ffff88003fa838c8  EFLAGS: 00000202
[  440.123003] RAX: 0000000000000101 RBX: ffff88003c8c3600 RCX: 0000000000000001
[  440.123003] RDX: 0000000000000101 RSI: 0000000000000001 RDI: ffff88003c8c3680
[  440.123003] RBP: 0000000000000001 R08: 0000000000000101 R09: 0000000000000000
[  440.123003] R10: 0000000000000010 R11: 0000000000000004 R12: 0000000000000002
[  440.123003] R13: ffff88002fcf8d00 R14: 0000000000000000 R15: ffff88003d3a5800
[  440.123003] FS:  00007f748d8a66a0(0000) GS:ffff88003fa80000(0000) knlGS:0000000000000000
[  440.123003] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  440.123003] CR2: 0000000000615510 CR3: 000000003c434000 CR4: 00000000000006e0
[  440.123003] Stack:
[  440.123003]  ffffffffa02c3451 ffff88003ab97e00 0000000100000001 ffff88003d568000
[  440.123003]  01ffc90002db5010 0000000000000004 dabab543e5e0a561 ffff88002fcf8d00
[  440.123003]  0000000000000002 ffff88002fcf8d00 ffff88003d3a5800 ffff88003ab97e00
[  440.123003] Call Trace:
[  440.123003]  <IRQ>
[  440.123003]  [<ffffffffa02c3451>] ? __imq_nf_queue+0x231/0x6f0 [imq]
[  440.123003]  [<ffffffffa02c3b42>] ? imq_nf_queue+0x232/0x376 [imq]
[  440.123003]  [<ffffffff814995a0>] ? ip_fragment.constprop.5+0x80/0x80
[  440.123003]  [<ffffffff8148e330>] ? nf_queue+0x130/0x190
[  440.123003]  [<ffffffff8148d396>] ? nf_hook_slow+0x56/0xd0
[  440.123003]  [<ffffffff8149a164>] ? ip_output+0xd4/0xf0
[  440.123003]  [<ffffffff814995a0>] ? ip_fragment.constprop.5+0x80/0x80
[  440.123003]  [<ffffffff814961da>] ? ip_forward+0x39a/0x490
[  440.123003]  [<ffffffff81495dd0>] ? ip_frag_mem+0x40/0x40
[  440.123003]  [<[  468.123003] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 22s! [swapper/1:0]
[  468.123003] Modules linked in: xt_recent xt_nat xt_set ip_set_bitmap_ipmac ip_set_hash_ip ip_set nfnetlink xt_IMQ cls_u32 sch_sfq sch_htb imq xt_ACCOUNT(O) ipt_REJECT nf_reject_ipv4 xt_tcpudp xt_comment xt_hashlimit xt_multiport xt_addrtype xt_conntrack xt_mark iptable_mangle iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 iptable_raw ip6table_filter ip6_tables iptable_filter ip_tables x_tables 8021q mrp garp stp llc ipv6 parport_pc parport k8temp powernow_k8 processor i2c_nforce2 i2c_dev i2c_core nf_nat_tftp nf_nat_snmp_basic nf_conntrack_snmp nf_conntrack_broadcast nf_nat_sip nf_nat_pptp nf_nat_proto_gre nf_nat_irc nf_nat_h323 nf_nat_ftp nf_nat_amanda nf_nat nf_conntrack_tftp nf_conntrack_sip nf_conntrack_pptp nf_conntrack_proto_gre nf_conntrack_irc nf_conntrack_h323 nf_conntrack_ftp ts_kmp nf_conntrack_amanda nf_conntrack ixgbe(O) hwmon dca vxlan udp_tunnel ip6_udp_tunnel e1000e(O) ptp pps_core e1000
[  468.123003] CPU: 1 PID: 0 Comm: swapper/1 Tainted: G O L  4.4.38-x86_64 #2
[  468.123003] Hardware name: System manufacturer System Product Name/M2N-XE, BIOS 0107    12/10/2007
[  468.123003] task: ffff88003e0f6180 ti: ffff88003e174000 task.ti: ffff88003e174000
[  468.123003] RIP: 0010:[<ffffffff81088910>] [<ffffffff81088910>] queued_spin_lock_slowpath+0x140/0x150
[  468.123003] RSP: 0018:ffff88003fa838c8  EFLAGS: 00000202
[  468.123003] RAX: 0000000000000101 RBX: ffff88003c8c3600 RCX: 0000000000000001
[  468.123003] RDX: 0000000000000101 RSI: 0000000000000001 RDI: ffff88003c8c3680
[  468.123003] RBP: 0000000000000001 R08: 0000000000000101 R09: 0000000000000000
[  468.123003] R10: 0000000000000010 R11: 0000000000000004 R12: 0000000000000002
[  468.123003] R13: ffff88002fcf8d00 R14: 0000000000000000 R15: ffff88003d3a5800
[  468.123003] FS:  00007f748d8a66a0(0000) GS:ffff88003fa80000(0000) knlGS:0000000000000000
[  468.123003] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[  468.123003] CR2: 0000000000615510 CR3: 000000003c434000 CR4: 00000000000006e0
[  468.123003] Stack:
[  468.123003]  ffffffffa02c3451 ffff88003ab97e00 0000000100000001 ffff88003d568000
[  468.123003]  01ffc90002db5010 0000000000000004 dabab543e5e0a561 ffff88002fcf8d00
[  468.123003]  0000000000000002 ffff88002fcf8d00 ffff88003d3a5800 ffff88003ab97e00
[  468.123003] Call Trace:
[  468.123003]  <IRQ>
[  468.123003]  [<ffffffffa02c3451>] ? __imq_nf_queue+0x231/0x6f0 [imq]
[  468.123003]  [<ffffffffa02c3b42>] ? imq_nf_queue+0x232/0x376 [imq]
[  468.123003]  [<ffffffff814995a0>] ? ip_fragment.constprop.5+0x80/0x80
[  468.123003]  [<ffffffff8148e330>] ? nf_queue+0x130/0x190
[  468.123003]  [<ffffffff8148d396>] ? nf_hook_slow+0x56/0xd0
[  468.123003]  [<ffffffff8149a164>] ? ip_output+0xd4/0xf0
[  468.123003]  [<ffffffff814995a0>] ? ip_fragment.constprop.5+0x80/0x80
[  468.123003]  [<ffffffff814961da>] ? ip_forward+0x39a/0x490
[  468.123003]  [<ffffffff81495dd0>] ? ip_frag_mem+0x40/0x40
[  468.123003]  [<ffffffff8148e467>] ? nf_reinject+0xd7/0x160
[  468.123003]  [<ffffffffa02c31a9>] ? imq_dev_xmit+0x89/0xc0 [imq]
[  468.123003]  [<ffffffff8145ad17>] ? dev_hard_start_xmit+0xa7/0x3e0
[  468.123003]  [<ffffffffa02c3408>] ? __imq_nf_queue+0x1e8/0x6f0 [imq]
[  468.123003]  [<ffffffffa02c3b42>] ? imq_nf_queue+0x232/0x376 [imq]
[  468.123003]  [<ffffffff81494160>] ? ip_local_deliver_finish+0x1d0/0x1d0
[  468.123003]  [<ffffffff8148e330>] ? nf_queue+0x130/0x190
[  468.123003]  [<ffffffff8148d396>] ? nf_hook_slow+0x56/0xd0
[  468.123003]  [<ffffffff814949b6>] ? ip_rcv+0x306/0x3a0
[  468.123003]  [<ffffffff81494160>] ? ip_local_deliver_finish+0x1d0/0x1d0
[  468.123003]  [<ffffffff81456989>] ? __netif_receive_skb_core+0x6d9/0xa20
[  468.123003]  [<ffffffff814590df>] ? netif_receive_skb_internal+0x2f/0xa0
[  468.123003]  [<ffffffff814592b5>] ? napi_gro_flush+0x55/0x70
[  468.123003]  [<ffffffff81459331>] ? napi_complete_done+0x61/0x90
[  468.123003]  [<ffffffffa003724f>] ? e1000e_write_itr+0x154f/0x1740 [e1000e]
[  468.123003]  [<ffffffff8145aa24>] ? net_rx_action+0x204/0x340
[  468.123003]  [<ffffffff81054f3a>] ? __do_softirq+0xfa/0x290
[  468.123003]  [<ffffffff81055205>] ? irq_exit+0x95/0xa0
[  468.123003]  [<ffffffff8150e53f>] ? do_IRQ+0x4f/0xd0
[  468.123003]  [<ffffffff8150cabf>] ? common_interrupt+0x7f/0x7f
[  468.123003]  <EOI>
[  468.123003]  [<ffffffff81087271>] ? cpu_startup_entry+0x111/0x330
[  468.123003]  [<ffffffff81036007>] ? start_secondary+0x137/0x160
[  468.123003] Code: 07 01 48 8b 02 48 85 c0 75 0a f3 90 48 8b 02 48 85 c0 74 f6 c7 40 08 01 00 00 00 e9 5f ff ff ff 83 fa 01 75 04 eb 11 f3 90 8b 07 <84> c0 75 f8 b8 01 00 00 00 66 89 07 c3 f3 c3 90 66 66 66 66 90

Add rcu protection to imq_dev_xmit()

(Using kernel 4.4.x currently)

nf_reinject() needs to be called with rcu lock held, as it calls rcu_dereference().

Otherwise it traps on a WARN_ON(!rcu_read_lock_held()) in dst.h when lockdep is enabled.

WARNING: CPU: 3 PID: 0 at include/net/dst.h:319 __xfrm_route_forward+0x138/0x170()

linux-imq-rcu.diff.txt

New patch for kernel 4.4.116 is need. Old will not longer fit

By the upgrade of the kernel-line 4.4 from version 4.4.115 to 4.4.116 are some deeper modifications be made from the kernel. With that the linuximq-patch will not longer fit here and need to be reworked.

nf_register_hook has been removed in kernel 4.13.8

See https://gitlab.collabora.com/martyn/linux/commit/cf56c2f892a8a1870a8358114ad896772da7543a

shaper linux-4.13.8-gentoo # make
CHK include/config/kernel.release
CHK include/generated/uapi/linux/version.h
CHK include/generated/utsrelease.h
CHK include/generated/bounds.h
CHK include/generated/timeconst.h
CHK include/generated/asm-offsets.h
CALL scripts/checksyscalls.sh
CHK scripts/mod/devicetable-offsets.h
CHK include/generated/compile.h
CHK kernel/config_data.h
CC drivers/net/imq.o
drivers/net/imq.c:765:14: error: initialization from incompatible pointer type [-Werror=incompatible-pointer-types]
.validate = imq_validate,
^
drivers/net/imq.c:765:14: note: (near initialization for ‘imq_link_ops.validate’)
drivers/net/imq.c: In function ‘imq_init_hooks’:
drivers/net/imq.c:778:8: error: implicit declaration of function ‘nf_register_hooks’ [-Werror=implicit-function-declaration]
ret = nf_register_hooks(imq_ops, ARRAY_SIZE(imq_ops));
^
drivers/net/imq.c: In function ‘imq_unhook’:
drivers/net/imq.c:915:2: error: implicit declaration of function ‘nf_unregister_hooks’ [-Werror=implicit-function-declaration]
nf_unregister_hooks(imq_ops, ARRAY_SIZE(imq_ops));
^
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:303: drivers/net/imq.o] Error 1
make[1]: *** [scripts/Makefile.build:561: drivers/net] Error 2
make: *** [Makefile:1019: drivers] Error 2

Lockdep workaround for recursive transmit lock usage

(Using imq on a 4.4.x kernel currently, but problem exists also for 3.16.x, and most likely everything since at least that)

When compiling the kernel with lockdep support, and using one IMQ device for incoming traffic and another for outgoing, lockdep thinks the IMQ transmit would be taken recursively, making it scream - but in reality only one transmit lock from each participating imq device is taken, not the same one multiple times.

Included patch works around this problem by marking them with lockdep_set_novalidate_class().

linux-imq-lockdep.diff.txt

iptables-restore cannot work with imq

Some rules just like:
-t mangle -A POSTROUTING -d 192.168.5.0/24 -j IMQ --todev 0

iptables-restore cannot work properly and throw the exception:

iptables-restore v1.4.7: unknown option --todev' Tryiptables-restore -h' or 'iptables-restore --help' for more information.

imq with Kernel 4.4.25 and grsecurity

compile kernel 4.4.25 with grsecurity and imq patch 4.4.5 failed.
compile kernel without grsecurity patch is working.

---snip---
drivers/net/imq.c:749:29: error: constified variable ‘imq_link_ops’ placed into writable section ".data..read_mostly"
static struct rtnl_link_ops imq_link_ops __read_mostly = {
^
scripts/Makefile.build:264: recipe for target 'drivers/net/imq.o' failed
make[2]: *** [drivers/net/imq.o] Error 1
scripts/Makefile.build:403: recipe for target 'drivers/net' failed
make[1]: *** [drivers/net] Error 2
Makefile:963: recipe for target 'drivers' failed
make: *** [drivers] Error 2
make: *** Waiting for unfinished jobs....
---snip---

imq with grsecurity compiles on kernel 4.4.21 without problems.

Do you have any plan to improve the performance of IMQ?

Hi ,

Firstly, thanks you for sharing the IMQ.
I did performance tests recently.
The HTTP RPS could reach 14 k/s without IMQ, but it drop down to 6 k/s with the following IMQ rules.

iptables -t mangle -A FORWARD -i lan1 -j IMQ --todev 0
iptables -t mangle -A FORWARD -i wan1 -j IMQ --todev 1

IMQ on kernel 3.10.98

Hi,

I'm running IMQ on kernel 3.10.98 via the patch http://linuximq.mantech.ro/patches/patch-imqmq-3.10.27.diff.xz. Everything seem to work fine. However, when loading traffic to the machine and running tcpdump on the 'any' interface (linux cooked mode) I get a lot of these messages in the kernel log:

[611512.049026] skbuff: IMQ: kfree_skb: skb->nf_queue_entry != NULL

I've reproduced the issue on different 3.10.x versions and on the 3.2.x kernel versions. Running tcpdump on a specific (imq enabled) interface doesn't introduce the error message. E.g. tcpdump -ni eth0.

The machine has been under heavy load with a running tcpdump on the 'any' interface for a week and the kernel doesn't panic but has yielded a lot of the above error message. Can I neglect the error message?

Any ideas?

Thanks and regards,

Erik

No working IMQ-patch for kernel-line 5.9

The IMQ-patch from kernel 5.4 will not work with Kernel-line 5.9
So it is need to rework/modify the patch.
This is an new issue identical as #89 for Kernel-line 5.7 and 5.8

imq patch 3.10 error during kernel compilation

I need to use older kernel as newer not properly limiting bandwidth but kernel is getting error during compilation
linux-3.10.10 with linux-3.10-imq.diff, patch apply ok but compilation with error

In file included from include/linux/kernel.h:13:0,
from include/linux/cache.h:4,
from include/linux/time.h:4,
from include/linux/stat.h:18,
from include/linux/module.h:10,
from net/core/skbuff.c:41:
net/core/skbuff.c: In function ‘skb_release_head_state’:
net/core/skbuff.c:596:5: warning: cast from pointer to integer of different size [-Wpointer-to-int-cast]
(unsigned int)skb->cb_next);
^
include/linux/printk.h:218:37: note: in definition of macro ‘pr_warning’
printk(KERN_WARNING pr_fmt(fmt), ##VA_ARGS)
^
net/core/skbuff.c:595:4: note: in expansion of macro ‘pr_warn’
pr_warn("IMQ: kfree_skb: skb->cb_next: %08x\n",
^
net/core/skbuff.c: In function ‘skb_init’:
net/core/skbuff.c:3129:14: error: invalid application of ‘sizeof’ to incomplete type ‘struct skb_cb_table’
sizeof(struct skb_cb_table),
^
net/core/skbuff.c: At top level:
net/core/skbuff.c:3473:13: warning: ‘skb_copy_stored_cb’ defined but not used [-Wunused-function]
static void skb_copy_stored_cb(struct sk_buff *new, const struct sk_buff *__old)
^
make[2]: *** [net/core/skbuff.o] Error 1
make[1]: *** [net/core] Error 2

the same with
linux-3.10.99 with linux-3.10-imq.diff
Please note also patch linux-3.10.27-imq.diff is malformed (look on patch size 4x bigger than others)
can anyone help ?

kernel 4.14

Is someone working on patch that will work for new LTS kernel ?

license issue

The project license was changed to GPLv3+ in this commit:.
But the linux kernel license is GPLv2-only.

missing MODULE_LICENSE

WARNING: modpost: missing MODULE_LICENSE() in drivers/net/imq.o

Kernel-Patch will not with Kernel-Line 5.7

The actually Kernel-Patch for Kernel 5.4 will not longer fir for Kernel 5.7.
So there is an update for this need.
There be problems by the patching self and also during the build as the variable nf_qeue_entry_release_refs is also changed in the Kernel (Something like static)

Fail to patch 4.4.162 with linux-4.4.32-imq.diff

Hi,
When patching the current linux kernel 4.4.162 it fails to patch, should we use patch 4.4-imq.diff?

Thanks,

Output:
linux-4.4.162]# patch -p1 < ../patches/linux-4.4.32-imq.diff
patching file drivers/net/imq.c
patching file drivers/net/Kconfig
patching file drivers/net/Makefile
patching file include/linux/imq.h
patching file include/linux/netdevice.h
Hunk #1 succeeded at 3413 (offset 16 lines).
patching file include/linux/netfilter/xt_IMQ.h
patching file include/linux/netfilter_ipv4/ipt_IMQ.h
patching file include/linux/netfilter_ipv6/ip6t_IMQ.h
patching file include/linux/skbuff.h
Hunk #2 succeeded at 572 (offset 1 line).
Hunk #3 succeeded at 584 (offset 1 line).
Hunk #6 succeeded at 3444 (offset 12 lines).
patching file include/net/netfilter/nf_queue.h
Hunk #1 succeeded at 31 with fuzz 2.
patching file include/net/pkt_sched.h
patching file include/net/sch_generic.h
patching file include/uapi/linux/netfilter.h
patching file net/core/dev.c
Hunk #2 succeeded at 2776 (offset 63 lines).
Hunk #3 succeeded at 2819 (offset 63 lines).
patching file net/core/skbuff.c
Hunk #4 succeeded at 3456 with fuzz 1 (offset 22 lines).
patching file net/ipv6/ip6_output.c
patching file net/netfilter/core.c
patching file net/netfilter/Kconfig
patching file net/netfilter/Makefile
patching file net/netfilter/nf_internals.h
patching file net/netfilter/nf_queue.c
Hunk #1 succeeded at 27 with fuzz 2 (offset -1 lines).
Hunk #2 succeeded at 131 (offset -2 lines).
Hunk #3 FAILED at 142.
Hunk #4 succeeded at 216 (offset -1 lines).
1 out of 4 hunks FAILED -- saving rejects to file net/netfilter/nf_queue.c.rej
patching file net/netfilter/xt_IMQ.c
patching file net/sched/sch_generic.c

linux-4.4.162]# cat net/netfilter/nf_queue.c.rej
--- net/netfilter/nf_queue.c 2016-01-11 05:01:32.000000000 +0600
+++ net/netfilter/nf_queue.c 2016-12-03 20:11:46.566088622 +0700
@@ -142,7 +160,17 @@ int nf_queue(struct sk_buff *skb,
const struct nf_queue_handler *qh;

/* QUEUE == DROP if no one is waiting, to be safe. */
- qh = rcu_dereference(queue_handler);
+ if (queuetype == NF_IMQ_QUEUE) {
+#if defined(CONFIG_IMQ) || defined(CONFIG_IMQ_MODULE)
+ qh = rcu_dereference(queue_imq_handler);
+#else
+ BUG();
+ goto err_unlock;
+#endif
+ } else {
+ qh = rcu_dereference(queue_handler);
+ }
+
if (!qh) {
status = -ESRCH;
goto err;

System hangs

We have replaced our core router with a new machine (+kernel and linux version) and now it hangs from time time.
It happens mostly when the script updates the shapers i.e. when tc adds qdiscs and classes to the imq device. Usually there is nothing in the logs and the changes are quite a lot, so it was difficult to narrow it down, but this time we got a call trace in the log and looks like imq (related) kernel bug:

[2663308.555346] rcu: INFO: rcu_sched self-detected stall on CPU
[2663308.555351] rcu: #0119-...!: (60000 ticks this GP) idle=02e/1/0x4000000000000002 softirq=1406852195/1406852195 fqs=0 last_accelerate: 86b2/7112, nonlazy_posted: 0, ..
[2663308.555351] rcu: #11 (t=60000 jiffies g=2405964101 q=974)
[2663308.555354] rcu: rcu_sched kthread starved for 60000 jiffies! g2405964101 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=8
[2663308.555354] rcu: RCU grace-period kthread stack dump:
[2663308.555355] rcu_sched I 0 10 2 0x80000000
[2663308.555356] Call Trace:
[2663308.555363] ? __schedule+0x225/0x850
[2663308.555364] schedule+0x32/0x80
[2663308.555365] schedule_timeout+0x176/0x360
[2663308.555368] ? __next_timer_interrupt+0xd0/0xd0
[2663308.555372] rcu_gp_kthread+0x404/0x830
[2663308.555373] ? sync_sched_exp_handler+0xa0/0xa0
[2663308.555376] kthread+0x117/0x130
[2663308.555377] ? kthread_create_worker_on_cpu+0x70/0x70
[2663308.555378] ret_from_fork+0x35/0x40
[2663308.555382] Sending NMI from CPU 9 to CPUs 1:
[2663308.555451] NMI backtrace for cpu 1
[2663308.555452] CPU: 1 PID: 17 Comm: ksoftirqd/1 Tainted: G O 4.19.75 #1
[2663308.555452] Hardware name: Supermicro Super Server/X11SPM-TPF, BIOS 3.0c 03/27/2019
[2663308.555452] RIP: 0010:queued_spin_lock_slowpath+0x138/0x1a0
[2663308.555453] Code: 85 c9 74 37 c7 41 08 01 00 00 00 65 ff 0d 7c 4c f2 6b c3 89 c2 30 e6 a9 00 00 ff ff 75 6b 85 d2 74 0e 8b 07 84 c0 74 08 f3 90 <8b> 07 84 c0 75 f8 b8 01 00 00 00 66 89 07 c3 f3 90 48 8b 0a 48 85
[2663308.555453] RSP: 0018:ffff9f138033ba80 EFLAGS: 00000202
[2663308.555453] RAX: 0000000000280101 RBX: 0000000000000001 RCX: 0000000000000000
[2663308.555454] RDX: 0000000000000001 RSI: 0000000000000000 RDI: ffff920469443080
[2663308.555454] RBP: 0000000000000000 R08: ffff9204ce589990 R09: 000000000003d08f
[2663308.555454] R10: 00000000fffffffe R11: 0000000000000001 R12: ffff92043a8f9900
[2663308.555454] R13: ffff920469443000 R14: ffff92043b60d000 R15: ffff920469445000
[2663308.555455] FS: 0000000000000000(0000) GS:ffff9204ef840000(0000) knlGS:0000000000000000
[2663308.555455] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[2663308.555455] CR2: 00007f8159cb3900 CR3: 00000002d2a0a006 CR4: 00000000007606e0
[2663308.555455] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[2663308.555456] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[2663308.555456] PKRU: 55555554
[2663308.555456] Call Trace:
[2663308.555456] __imq_nf_queue+0x3ab/0x780 [imq]
[2663308.555456] ? ipt_do_table+0x35c/0x620 [ip_tables]
[2663308.555456] imq_nf_queue+0x1f3/0x30b [imq]
[2663308.555457] ? kmem_cache_alloc_trace+0x43/0x1d0
[2663308.555457] nf_queue+0x158/0x2e0
[2663308.555457] nf_hook_slow+0x8a/0xf0
[2663308.555457] ip_output+0xd7/0xf0
[2663308.555457] ? __ip_flush_pending_frames.isra.0+0x80/0x80
[2663308.555458] ip_forward+0x3fe/0x460
[2663308.555458] ? ip_defrag.cold+0x37/0x37
[2663308.555458] ip_rcv+0xb5/0xc0
[2663308.555458] ? ip_rcv_finish_core.isra.0+0x380/0x380
[2663308.555458] __netif_receive_skb_one_core+0x53/0x70
[2663308.555459] netif_receive_skb_internal+0x55/0x100
[2663308.555459] napi_gro_receive+0x5a/0xf0
[2663308.555459] +0x8dd/0x15d0 [i40e]

I have found this bug ( https://bugzilla.kernel.org/show_bug.cgi?id=205025 ) which looks similar, but it is for 5.3 while my kernel is 4.19.75
I have found this link ( https://stackoverflow.com/questions/51236711/is-it-safe-to-use-rcu-dereference-inside-local-bh-disable-local-bh-enable ), which states that when rcu_read_lock_bh is used rcu_dereference_bh should be used, while in imq there is "q = rcu_dereference(txq->qdisc);" ... can this be the cause for the hang?

DELETE

Delete

Patch won`t apply on stable 4.1.12

latest linux-4.1-imq.patch fails:

oot@rwl0:/home/kernel/linux-4.1.12# patch -p1 < ../linux-4.1-imq.diff
patching file drivers/net/Kconfig
patching file drivers/net/Makefile
patching file drivers/net/imq.c
patching file include/linux/imq.h
patching file include/linux/netdevice.h
patching file include/linux/netfilter/xt_IMQ.h
patching file include/linux/netfilter_ipv4/ipt_IMQ.h
patching file include/linux/netfilter_ipv6/ip6t_IMQ.h
patching file include/linux/skbuff.h
Hunk #6 succeeded at 3233 (offset -1 lines).
patching file include/net/netfilter/nf_queue.h
Hunk #1 succeeded at 31 (offset 2 lines).
patching file include/net/pkt_sched.h
patching file include/uapi/linux/netfilter.h
patching file net/core/dev.c
Hunk #2 succeeded at 2645 (offset -4 lines).
Hunk #3 succeeded at 2688 (offset -4 lines).
Hunk #4 succeeded at 2777 (offset -4 lines).
patching file net/core/skbuff.c
Hunk #4 succeeded at 3450 (offset 1 line).
patching file net/ipv6/ip6_output.c
patching file net/netfilter/Kconfig
patching file net/netfilter/Makefile
patching file net/netfilter/core.c
Hunk #1 succeeded at 179 (offset 1 line).
patching file net/netfilter/nf_internals.h
Hunk #1 FAILED at 18.
1 out of 1 hunk FAILED -- saving rejects to file net/netfilter/nf_internals.h.rej
patching file net/netfilter/nf_queue.c
Hunk #2 succeeded at 146 (offset 17 lines).
Hunk #3 succeeded at 157 (offset 17 lines).
Hunk #4 succeeded at 253 (offset 17 lines).
patching file net/netfilter/xt_IMQ.c
patching file net/sched/sch_generic.c

kernel-panic by kernel 4.14 when linuximq should start.

Hello,
with the kernel 4.14 i will get during the startup of my system an kernel-panic (As already told you for the kernel 4.14 series -> #66)
I figured now out the exactly point, when the fault comes up.
The commands will be:
tc qdisc add dev imq0 root handle 10: htb default 6 r2q 1
tc class add dev imq0 parent 10: classid 10:7 htb rate 50000kbit
tc qdisc add dev imq0 parent 10:7 sfq perturb 10
tc qdisc add dev imq1 root handle 20: htb default 4 r2q 1
tc class add dev imq1 parent 20: classid 20:7 htb rate 10000kbit
tc qdisc add dev imq1 parent 20:7 sfq perturb 10
tc class add dev imq1 parent 20:7 classid 20:1 htb rate 200kbit ceil 10000kbit prio 0
(Normaly there will follow some more lines from me configuration)

and exactly when the last line will be activated, the kernel-panic appears.
I use now also iproute2 in version 4.14.1 together with an kernel 4.14.14.

Has someone an identical problem and has maybe solved this already?
It will be great if there come some hints to me, how to solve this.

IMQ conflict with TPROXY

The TPROXY does not work if I put an IMQ rule before the TPROXY rules.

iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j IMQ --todev 0
iptables -t mangle -A PREROUTING -p tcp -m tcp --dport 80 -j TPROXY --on-ip 0.0.0.0 --on-port 8080 --tproxy-mark 1/1
iptables -t mangle -A PREROUTING -p tcp -m tcp --sport 80 -j MARK --set-xmark 0x1/0x1

ifb or imq with cake sqm?

I am curious as to how well imq scales with cake, especially multicores. I have been trying to find ways to improve inbound shaping in cake for a while.

https://docs.google.com/document/d/1tTYBPeaRdCO9AGTGQCpoiuLORQzN_bG3TAkEolJPh28/edit#heading=h.vbbnfu73wlpp

IMQ crashing and rebooting router

I have installed IMQ on my router and set up a few rules, but the moment the imq0 interface goes up the router reboots . The only info i can see in the netconsole and syslog is :
Oct 17 21:55:29 meumodem.lan MeuNovoModem: kernel: [ 165.870000] skbuff: IMQ: kfree_skb: skb->cb_next: 862cad80
Oct 17 21:55:29 meumodem.lan MeuNovoModem: kernel: [ 165.880000] skbuff: IMQ: kfree_skb: skb->nf_queue_entry != NULL
Oct 17 21:55:30 meumodem.lan MeuNovoModem: kernel: [ 166.120000] skbuff: IMQ: kfree_skb: skb->cb_next: 85aee340
Oct 17 21:55:30 meumodem.lan MeuNovoModem: kernel: [ 166.130000] skbuff: IMQ: kfree_skb: skb->nf_queue_entry != NULL
Oct 17 21:55:30 meumodem.lan MeuNovoModem: kernel: [ 166.170000] skbuff: IMQ: kfree_skb: skb->cb_next: 85aee440
Oct 17 21:55:30 meumodem.lan MeuNovoModem: kernel: [ 166.180000] skbuff: IMQ: kfree_skb: skb->nf_queue_entry != NULL
Oct 17 21:55:30 meumodem.lan MeuNovoModem: kernel: [ 166.420000] skbuff: IMQ: kfree_skb: skb->cb_next: 85aeeb00
Oct 17 21:55:30 meumodem.lan MeuNovoModem: kernel: [ 166.430000] skbuff: IMQ: kfree_skb: skb->nf_queue_entry != NULL
Oct 17 21:55:30 meumodem.lan MeuNovoModem: kernel: [ 166.880000] skbuff: IMQ: kfree_skb: skb->cb_next: 83014580
Oct 17 21:55:30 meumodem.lan MeuNovoModem: kernel: [ 166.880000] skbuff: IMQ: kfree_skb: skb->nf_queue_entry != NULL

uname -a

Linux MeuNovoModem 3.18.21 #7 Fri Oct 02 11:04:58 GMT 2015 mips GNU/Linux

The router has 128Mb RAM, and just one user and it reboots almost idle.

Provide patch for linux-3.14.5-hardened-r2 (3.14.x)

Request to provide a Linux IMQ patch for Hardened sources v3.14.5 (Gentoo Linux).

Kernel 4.9 and imq

Hi any news or free time to resync patch for Kernel 4.9

Please provide patch for new vanilla kernels 3.2 because old dont work

Error with kernel-4.8.0 and 4.8.1

Patch for kernel 4.8 seems to be incomplete:

ERROR: "validate_xmit_skb_list" [drivers/net/imq.ko] undefined!
ERROR: "dev_hard_start_xmit" [drivers/net/imq.ko] undefined!
make[1]: *** [scripts/Makefile.modpost:91: __modpost] Error 1
make: *** [Makefile:1180: modules] Error 2

Both functions are defined in netdevice.h and implemented in net/core/dev.c, but they are not visible.
I had to add EXPORT_SYMBOL(validate_xmit_skb_list); and EXPORT_SYMBOL(dev_hard_start_xmit);in dev.c

Well, my c/c++ knowlegde are basics, kernel experience - a big zero.
I just compared versions from 4.6 to 4.8 and dig on internet to understand notions new to me, so please advise me. The kernel is compiled, both bzImage and all modules, but it is safe ?

imq doesn't work on linux-3.14.77

hi,imq team:
I use imq to test QOS,and I was stacked with the issue:
I use the iptable-1.4.10 with imq patch, and kernel-3.14.77 with imq patch;
and set the rule to test QOS, but it doesn't work. it is strange.
it seems the data is goto the imq0 a little, and most data goto the lan directly.
maybe it is releated to netfilter(kernel) or some buffer code?
Any advices?

Possible memory leak imq patch for kernel 4.8.4

Hello, in history i used kernel 4.2.3 with imq patch (only) for 1 year and it work fine. Before 2 days i installed on same router (debian based) kernel 4.8.4 patched with imq (patch founded in closed issue #46). After one day router suddenly rebooted (kernel 4.8.4-imq, 2GB ram). The traffic is about 150Mbps. On my graph i see that it use all memory (about 100MB/hour). Router have 2GB of ram. After this first reboot i try diagnose what happen. Router using more and more memory. But no process use the memory!

router6:~# ps aux | awk '{sum+=$6} END {print sum / 1024}'
85.8398
router6:~# free -m
             total       used       free     shared    buffers     cached
Mem:          1985       1939         46          0          0         41
-/+ buffers/cache:       1897         88
Swap:            0          0          0

I try many thing to analyse what happen, i cannot find what use the memory. Trying restarting services and still same. Memory is exhausting. When i stop imq with "ifconfig imq0 down", exhausting of memory stop! The current state is that router have 2GB, processes used 85MB and 46M is free, so the kernel use 1917M of memory (2048-85-46)

For all information, i have same kernel on other 3 router (with other hardware), and there is no problem with memory leak or using by imq. Used memory on other router with same kernel (4.8.4-imq) is about 400MB in 2GB RAM. I know, it sound strange. If you want, i can make some other test or append some diagnostic output. I could not experiment too much because router is in network with customers.

imq / linuximq Goto Github PK

linuximq's People

Stargazers

Watchers

Forkers

linuximq's Issues

uname -a

Recommend Projects

Recommend Topics

Recommend Org