libos-nuse / net-next-nuse Goto Github PK

ideas:

1. proxy-based solution like microkernel server
   the server accept and dispatch which NUSE process is listening
2. .conf based solution
3. per-NUSE process UNIX domain socket (e.g., /var/run/nuse-1234-pid/RT_NETLINK)

currently, NUSEVIF (vif type) and nuse-(ifname) environmental variable is used.

when using select with timeout on nuse process, duration of timeout is getting longer...

nginx-1.6.1 crashes when epoll_wait () wakes up second time.

Hi,
I was trying to test nuse but could not compile at all. It complains that this directory is missing and attempts to fetch it while "make library ARCH=lib" fail with error:
Cloning into 'arch/lib/tools'...
remote: Counting objects: 324, done.
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed
make: *** [arch/lib/tools] Error 128

I would appreciate any help on this.

currently, all interface name of VIF needs to include a local one:
e.g.,
if
% ifconfig
lo:
eth0
p1p1:

then, vif need to be assign nuse-eth0 or nuse-p1p1.
but an interface created for tap doesn't need to be like this: this restriction should be relaxed.

only PORTID=0 is available now.

it's like rumpserver implementation, but a built-in server in NUSE process.

Discussed here: https://groups.google.com/g/ns-3-users/c/Lyc6H3ONVoM

GitHub policy change: https://github.blog/2021-09-01-improving-git-protocol-security-github/

% sudo ./nuse ping www.google.com
ping: unknown host: www.google.com

After make defconfig ARCH=lib && make library ARCH=lib, examining liblinux.so's program headers with readelf -l shows:

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x00000000005eabf4 0x00000000005eabf4  **RWE**    200000
  LOAD           0x0000000000600000 0x0000000000800000 0x0000000000800000
                 0x000000000003b3f0 0x000000000004efe8  RW     200000
  DYNAMIC        0x000000000060f460 0x000000000080f460 0x000000000080f460
                 0x0000000000000180 0x0000000000000180  RW     8
  NOTE           0x00000000000001c8 0x00000000000001c8 0x00000000000001c8
                 0x0000000000000024 0x0000000000000024  R      4
  GNU_EH_FRAME   0x000000000057c050 0x000000000057c050 0x000000000057c050
                 0x000000000001600c 0x000000000001600c  R      4
  GNU_STACK      0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000  RW     10
  GNU_RELRO      0x0000000000600000 0x0000000000800000 0x0000000000800000
                 0x0000000000011000 0x0000000000011000  R      1

 Section to Segment mapping:
  Segment Sections...
   00     .note.gnu.build-id .gnu.hash .dynsym .dynstr .rela.dyn .rela.plt .init .plt .text .init.text .sched.text .ref.text .exit.text .fini .rodata __ksymtab_strings .init.rodata .eh_frame_hdr .eh_frame

The issue is the rwx permissions on section 00. As far as I can tell, the ELF standard allows for executable regions to be writable, but this breaks some standard tools (in particular, Valgrind on amd64) and may cause very subtle bugs down the line. It will also cause issues if nuse is run on architectures with W^X protection. It is caused by a quirk of config options, and isn't necessary for the library to work.

The problem is in section .init.rodata, which has rw- permissions. The problem in .init.rodata comes from net/ipv6/{fib6_rules,addrlabel}.o and net/ipv4/{fib_rules,ipmr}.o, which all have .init.rodata sections with rw- permissions.

The problem there comes from the use of the following macros, defined in include/net/net_namespace.h:

#ifdef CONFIG_NET_NS
#define __net_init
#define __net_exit
#define __net_initdata
#define __net_initconst
#else
#define __net_init  __init
#define __net_exit  __exit_refok
#define __net_initdata  __initdata
#define __net_initconst __initconst
#endif

which depend on the macros defined in include/linux/init.h:

#define __init          __section(.init.text) __cold notrace
#define __initdata      __section(.init.data)
#define __initconst     __constsection(.init.rodata)
#define __exitdata      __section(.exit.data)
#define __exit_call     __used __section(.exitcall.exit)

It can be fixed either by changing the #ifdef CONFIG_NET_NS line in include/net/net_namespace.h to #if 1, or by adding:

config NET_NS
       def_bool y

to arch/lib/Kconfig. Then As far as I can tell, CONFIG_NET_NS=y doesn't break anything in nuse, but I haven't yet tested it thoroughly. There may be a better fix that involves changing the problematic functions annotated with __net_initconst in net/ipv6/{fib6_rules,addrlabel}.c and net/ipv4/{fib_rules,ipmr}.c --- presumably this is not intended behavior.

hijack code are duplicated between rumpclient (hijack.c) and nuse-glue.c. it should be unified for the future.

currently, crashed on Travis CI (https://travis-ci.org/thehajime/net-next-nuse/builds/41203974).

I saw the slides Userspace networking: Network Stack in Userspace which say ping and iperf work but nc, wget, dig, and host do not work.

Why do they not work, where is the technical reason? Is it because of unimplemented syscalls they require?

Also, congratulations, your code works on my Debian 8.0. I can ping google.com.

reported by Richard Weinberger on https://lkml.org/lkml/2015/3/30/31

https://lkml.org/lkml/2015/3/31/101

like 'linux ip=dhcp'.

from getenv(2) ?

• How to reproduce it
  • with raw socket

sudo NUSEDEV=eno16777736 nuse-eno16777736=192.168.209.39 ./nuse ../nginx-1.6.1/objs/nginx -c >pwd/../nginx-1.6.1/conf/nginx.conf -p ./

• with tap

sudo NUSEDEV=eth1 NUSEVIF=TAP nuse-eth1=192.168.0.2 ./nuse nginx -c pwd/../nginx-1.6.1/conf/nginx.conf -p ./

both are closing socket for VIF, resulting no VIF channel after that (no RX/TX).

Being as the project is in its initial stages, most of the talk is (necessarily) technical. In my view this project could be the road to the Holy Grail of protocol-free Purely Software Networking. See Tearing Down the Protocol Wall on my LinkedIn page.
I have some ideas and proposals which are burning my insides. Where can we discuss them?

TSIA.

[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `ping 192.168.49.58'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  __list_del (next=0x7f759c713e1a <lib_sock_poll+90>, prev=0x0) at ./include/linux/list.h:90
90              prev->next = next;
(gdb) bt
#0  __list_del (next=0x7f759c713e1a <lib_sock_poll+90>, prev=0x0) at ./include/linux/list.h:90
#1  __list_del_entry (entry=0x7fff7c482948) at ./include/linux/list.h:102
#2  list_del_init (entry=0x7fff7c482948) at ./include/linux/list.h:145
#3  autoremove_wake_function (wait=0x7fff7c482930, mode=<optimized out>, sync=<optimized out>, key=<optimized out>) at arch/lib/sched.c:173
#4  0x00007f759c714bc6 in __wake_up (q=0x1645318, mode=mode@entry=1, nr_exclusive=nr_exclusive@entry=1, key=key@entry=0xc3) at arch/lib/sched.c:242
#5  0x00007f759c714c05 in __wake_up_sync_key (q=<optimized out>, mode=mode@entry=1, nr_exclusive=nr_exclusive@entry=1, key=key@entry=0xc3) at arch/lib/sched.c:251
#6  0x00007f759c75a2a9 in sock_def_readable (sk=0x16474f8) at net/core/sock.c:2235
#7  0x00007f759c7590ed in sock_queue_rcv_skb (sk=sk@entry=0x16474f8, skb=skb@entry=0x7f75800011b8) at net/core/sock.c:474
#8  0x00007f759c7e863c in raw_rcv_skb (sk=sk@entry=0x16474f8, skb=skb@entry=0x7f75800011b8) at net/ipv4/raw.c:315
#9  0x00007f759c7e9924 in raw_rcv (sk=sk@entry=0x16474f8, skb=0x7f75800011b8) at net/ipv4/raw.c:334
#10 0x00007f759c7e9b11 in raw_v4_input (skb=skb@entry=0x7f758c0008c8, iph=0x7f758c0009e6, hash=hash@entry=1) at net/ipv4/raw.c:194
#11 0x00007f759c7e9b6c in raw_local_deliver (skb=skb@entry=0x7f758c0008c8, protocol=protocol@entry=1) at net/ipv4/raw.c:216
#12 0x00007f759c7bea99 in ip_local_deliver_finish (sk=sk@entry=0x0, skb=skb@entry=0x7f758c0008c8) at net/ipv4/ip_input.c:203
#13 0x00007f759c7bf092 in NF_HOOK_THRESH (thresh=-2147483648, okfn=0x7f759c7be9a0 <ip_local_deliver_finish>, out=0x0, in=<optimized out>, skb=0x7f758c0008c8, sk=0x0, hook=1, pf=2 '\002')
    at ./include/linux/netfilter.h:220
#14 NF_HOOK (okfn=0x7f759c7be9a0 <ip_local_deliver_finish>, out=0x0, in=<optimized out>, skb=0x7f758c0008c8, sk=0x0, hook=1, pf=2 '\002') at ./include/linux/netfilter.h:242
#15 ip_local_deliver (skb=0x7f758c0008c8) at net/ipv4/ip_input.c:256
#16 0x00007f759c7bf2f3 in NF_HOOK_THRESH (thresh=-2147483648, okfn=0x7f759c7bebd0 <ip_rcv_finish>, out=0x0, in=0x163f200, skb=0x7f758c0008c8, sk=0x0, hook=0, pf=2 '\002')
    at ./include/linux/netfilter.h:220
#17 NF_HOOK (okfn=0x7f759c7bebd0 <ip_rcv_finish>, out=0x0, in=0x163f200, skb=0x7f758c0008c8, sk=0x0, hook=0, pf=2 '\002') at ./include/linux/netfilter.h:242
#18 ip_rcv (skb=<optimized out>, dev=0x163f200, pt=<optimized out>, orig_dev=<optimized out>) at net/ipv4/ip_input.c:455
#19 0x00007f759c76d203 in __netif_receive_skb_core (skb=0x7f758c0008c8, pfmemalloc=<optimized out>) at net/core/dev.c:3895
#20 0x00007f759c76da74 in process_backlog (napi=0x7f759cc7be70 <softnet_data+112>, quota=64) at net/core/dev.c:4506
#21 0x00007f759c76d8be in napi_poll (n=0x7f759cc7be70 <softnet_data+112>, repoll=repoll@entry=0x7f759b63ce20) at net/core/dev.c:4744
#22 0x00007f759c76db98 in net_rx_action (h=<optimized out>) at net/core/dev.c:4809
#23 0x00007f759c713fe3 in do_softirq () at arch/lib/softirq.c:69
#24 0x00007f759c714048 in softirq_task_function (context=<optimized out>) at arch/lib/softirq.c:28
#25 0x00007f759c45624d in nuse_task_start_trampoline (context=0x15a6250) at nuse.c:175
#26 0x00007f759b84e182 in start_thread (arg=0x7f759b63d700) at pthread_create.c:312
#27 0x00007f759bf7d47d in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:111

• hardware/or VM configuration (number of cpus, memory)
  • 8 cores Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz, 16G Memory
  • running Centos 6.4 final
  • running KVM to host multiple virtual guest
  • libos is on virtual guest running ubuntu 14.04.2 with 4 virtual cpus and 4G memory
• host OS (distribution, version, kernel version)
  • KVM guest OS 4 virtual cpus, 4G memory
  • Linux ubuntu14 3.16.0-30-generic #40~14.04.1-Ubuntu SMP Thu Jan 15 17:43:14 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux
  • libos is from your github clone on kernel 4.1.0-rc7+ :

# ./nuse ping 192.168.49.254

<5>Linux version 4.1.0-rc7+ (root@ubuntu14) (gcc version 4.8.2 (Ubuntu 4.8.2-19ubuntu1) ) #0 Wed Jun 17 12:56:11 PDT 2015

• how to build it (only make library ARCH=lib or with OPT=no)
  • make defconfig ARCH=lib
  • make library ARCH=lib
• how often this happens
  • once in two runs,

nuse-test.sh fails due to missing symbol, on Ubuntu 14.04 64bit.
Ubuntu 14.04.1 64bits works fine though.

ping: symbol lookup error: /usr/lib/libnuse-linux.so: undefined symbol: atexit

support vnstat (http://humdi.net/vnstat/)

at least, we need the followings:

• /proc like interface for NUSE
• counting packet statistics for sim-device.c

• mixture of user and kernel space headers
• duplicated prototype declaration across files
• coding style => should be linux net-next
• Werror compile

There is a typo in the net-next-nuse/include/net/cfg80211.h file:

I created a dedicated topic branch for this issue.

https://github.com/libos-nuse/net-next-nuse/tree/kbuild-makefile

I'm also working to solve this issue, patches are highly appreciated !!!

the specific challenges here for LibOS to be on Kbuild system are:

• we need to pick specific object files which is needed for our LibOS
  line 58 of arch/lib/Makefile (https://github.com/libos-nuse/net-next-nuse/blob/kbuild-makefile/arch/lib/Makefile#L58) is the part of this selection.
• usual conventions like core-y or obj-y is not able to use as-is
• we don't have a system call table at this moment: scripts/checksyscalls.sh reports tons of warnings

reference:
http://www.spinics.net/lists/linux-mm/msg87029.html

Linux sysctl parses files under /proc/sys/ so, redirecting open(2) for the PATH would be the first step.

with xmit_more API?

• http://netoptimizer.blogspot.jp/2014/10/unlocked-10gbps-tx-wirespeed-smallest.html
• https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=53fda7f7f9e8
• https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/include/linux/netdevice.h?id=71ae8f5271b31da1172751059deb8bfc32b2b759#n3468
• https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/net/core/dev.c?id=ce93718fb7cdbc064c3000ff59e4d3200bdfa744#n2556
• https://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/tree/net/core/pktgen.c?id=38b2cf2982dc73d3f07fe84fec8cc4ed9f64c1c5#n3367

all the stderr related print got no output.
fprintf(stderr), perror()

Is there a possibility to cross compile this to arm64 arch?

current system call proxy uses rumpkernel system call template, which derived from NetBSD system calls. It should be from Linux's one.

it returns an incorrect ifname.
using ptrace would be one of solution for that.

Hi, I am doing something with LibOS. And I am facing some problems which is I can't use it successfully with netmap.
When I want to test NUSE, I always meet a core dump after it shows the vif log just like this

create vif eth0
  address = 192.168.108.136
  netmask = 255.255.255.0
  macaddr = 00:0c:29:70:93:12
  type    = 1
Segmentation fault (core dumped)

My test conf as this

interface eth0
    address 192.168.108.136
    netmask 255.255.255.0
    macaddr 00:0c:29:70:93:12
#   if macaddr is not specified, random mac addr is used.
    viftype NETMAP


route
    network 192.168.108.0
    netmask 255.255.255.0
    gateway 192.168.108.2

While, I have already loaded the module netmap.ko and its driver e1000.ko

sunlifei@sunlifei-VPC:~/Documents/net-next-nuse/arch/lib/tools$ lsmod | grep netmap
netmap                114576  1 e1000

I'm confused what was wrong with my environment. Should it be something about the kenel version where I use

sunlifei@sunlifei-VPC:~/Documents/net-next-nuse/arch/lib/tools$ uname -a
Linux sunlifei-VPC 3.13.0-45-generic #1 SMP Mon Apr 27 22:00:27 CST 2015 x86_64 x86_64 x86_64 GNU/Linux

,would this matter?

currently, atexit(3) is nop. no gcda outputs even built with COV=yes.

current arch/sim does not represent the feature. maybe arch/lib is more appropriate ?

Hit this patch。
I use a wireless network card of laptop，Cannot connect to 5.8GWiFi of qca9887 with openwrt route
can get the IP address, but there is no network，but Mobile normal。

I capture wireless analysis No reply arp of station arp request。
Do you have any suggestions for me?
thanks ；

Hi all,

I am trying to install DCE with Linux Stack in order to execute some mmWave experiments, but I keep getting error messages. The last one failed in the addition of the following files:

In file included from rump/lib/librumpuser/rumpuser_port.h:70:0,
from rump/lib/librumpuser/rumpuser_sp.c:37:
./rump/sys/sys/cdefs.h:58:27: fatal error: machine/cdefs.h: No such file or directory

Could anyone help me?

Best regards,
José Craibas

I am following the manual and trying to run ns3 with mptcp kernel. And I notice that there is a branch in libos named mptcp_trunk_libos but that is a very old version of mptcp.
I am wondering if I can use a higer version of mptcp to generate the libsim_linux.so?

btw is this way here still works because I got errors when following it.

Good day ;-)

Hi,

I am using ubuntu16.04 to build mptcp_trunk_libos following the wiki but fail with following error:

  CC       rump/lib/librumpuser/rumpuser_sp.o
In file included from rump/lib/librumpuser/rumpuser_port.h:70:0,
                 from rump/lib/librumpuser/rumpuser_sp.c:37:
./rump/sys/sys/cdefs.h:58:27: fatal error: machine/cdefs.h: No such file or directory
compilation terminated.
Makefile.rump:79: recipe for target 'rump/lib/librumpuser/rumpuser_sp.o' failed
make[2]: *** [rump/lib/librumpuser/rumpuser_sp.o] Error 1
Makefile:88: recipe for target 'librumpserver.so' failed
make[1]: *** [librumpserver.so] Error 2
arch/lib/Makefile:193: recipe for target 'arch/lib/tools' failed
make: *** [arch/lib/tools] Error 2

It seems that there is no such a file in my whole system. How to solve it?

title says its all.