sysprog21 / semu Goto Github PK

Hi,

I just ported your great little emu to a very unlikely architecture and whilst doing so, I had to tweak the MMU code to go through the RAM access methods as well. I propose the code would be a lot more portable if MMU RAM access calls corresponding methods/functions as well, so that RAM access can really be anything. The current way of having a pointer into emu RAM is a bit of a breach of abstraction IMO.

I further noticed that every CPU instruction fetch goes through the MMU page walk and a very simple "1-entry-1-page" cache (one for INSN fetch, load & store) speeds up the emulation considerably. For the standard PC build, I got from about 10s to boot the emulated Linux to about 7s just by caching the current page for mmu_fetch.

As my code to do so is quite peculiar and adapted to the odd target architecture, I don't think it really fits well as an upstream patch. Still, I like to raise the issue of the MMU being direct-access here.

At present, initrd is missing, which means users have to boot Linux with initramfs enabled. This might not be convenient for someone who expects to specify a customized userland and/or benchmark suite at an early stage.

Expected output:

1. specify initrd-start and initrd-end in device tree.
2. Add an option -i to specify initrd image along with the functionality to load.

Reference:

• yariscv
• kvm-host

教授威武, 真的要給個Like你們 @jserv

Hello,
I would like to integrate riscv-tests into semu project.

Which is recommended way to implement test into semu?

1. Assign only one test binary to semu. The test command would be

   ./semu riscv-tests ./test/riscv-tests/rv64ui_p_add
   

   The implementation is trivial, just read binary and allocate CPU one time like current semu does.
   But we need to type specific command for each test.

2. Assign a test folder to semu. Semu has a loop to execute test binaries one by one in that folder.

   ./semu riscv-tests ./test/riscv-tests/
   

   In this case, we can create another C file to maintain all the test cases
   But we also need to consider allocating/freeing CPU resources per test case, right? Including bus, UART pthread...
   It would be more complicated.

   The pseudo code would be:

   for test in riscv-tests
       Open and read one test binary
       Allocate new CPU, which includes RAM, bus, UART pthread, etc
       Start CPU with limited loop round
       Check if register a0 equals zero and show test result
       Free CPU
   end
   

The virtio-pmem offers a virtual device that emulates a persistent memory device accessible by bytes. It delivers the disk image to the guest through a memory-mapped view of the file, allowing for direct mapping into the guest's address space if both the guest's operating system and filesystem are DAX compatible. The Linux virtio-pmem driver, activated by the CONFIG_VIRTIO_PMEM setting, makes pmem devices visible as /dev/pmem0, /dev/pmem1, and so on. These devices can be mounted like any regular block device. The riscv_em project includes support for a pmem-based ramdisk. This setup makes the filesystem accessible as /dev/pmem0, which can be mounted from userspace or designated as the root filesystem. It's important to note that this approach bypasses the initrd or initramdisk mechanisms due to their significant size constraints, which involve unpacking/copying the entire filesystem during boot.

During emulation, the semu_start() function initializes a single HART with a hartid of 0 and a start_addr value of (RAM_SIZE - 1024 * 1024). This means that there is only one HART present throughout the emulation process.

However, HSM (Hart Stete Management) is a mechanism used to manage the state of processor cores and is commonly applied in multi-core processor systems. It involves saving and restoring the HART's register state, program counter, and other contents when switching between different tasks or processes.

In a single-core processor system (only one HART), frequent HART state switching is not required.

Expectations:
Implement multi-core RISC-V processor system followed HSM State Machine

Validation:
check cat/proc/cpuinfo, currently as follows:

# cat /proc/cpuinfo
processor	: 0
hart		: 0
isa		: rv32ima
mmu		: sv32
mvendorid	: 0x12345678
marchid		: 0x80000001
mimpid		: 0x1

WebAssembly is capable to port rv32emu to browser. Then, I would like to test the WebAssembly translation of the semu codebase to see if kernels (like Linux or xv6) can be booted in a browser in order to facilitate the further integration of semu to rv32emu.

I would like to start with the smaller codebase and boot xv6 first. A few technical points are as follows:

1. It is necessary to simulate a new shell that is operating in a web frontend (Xterm.js is a good solution), after which the command is forwarded to the kernel's running shell. Some of the scope issues with the variables generated by Emscripten are confusing to me, so that I have implemented a hardcoded ls command to confirm the kernel is booted successfully. The demo below shows Emscripten's capability to boot a kernel.
2. The sh shell's dollar sign ($) of xv6 does not flush after exec, possibly some source kernel code needs to be changed.

Demo: xv6

You shall see output of ls command like below:

$ .              1 1 1024
..             1 1 1024
README         2 2 2059
cat            2 3 25016
echo           2 4 23800
forktest       2 5 13688
grep           2 6 28568
init           2 7 24896
kill           2 8 23752
ln             2 9 23704
ls             2 10 27320
mkdir          2 11 23856
rm             2 12 23840
sh             2 13 43824
stressfs       2 14 24864
usertests      2 15 159064
grind          2 16 39424
wc             2 17 26208
zombie         2 18 23224
console        3 19 0

Next, command bridging from web frontend shell(Xterm.js) to real shell will be solved, and then boot the Linux kernel.

akihikodaki's patch set includes support for vmnet which offers more flexibility than -netdev user, and allows higher network throughput. (see akihikodaki/qemu@72a35bb).

Take QEMU for example. To enable bridge mode, replace:

    -device virtio-net-pci,netdev=net \
    -netdev user,id=net,ipv6=off \

with

    -netdev vmnet-macos,id=n1,mode=bridged,ifname=en0 \
    -device virtio-net,netdev=n1 \

vmnet also offers "host" and "shared" networking model:

   -netdev vmnet-macos,id=str,mode=host|shared[,dhcp_start_address=addr,dhcp_end_address=addr,dhcp_subnet_mask=mask]

caveats:

1. vmnet requires running qemu as root, for now.
2. current vmnet API (Apple) doesn't support setting MAC address, so it will be randomized every time the VM is started.
   To work around 2), for now it's possible to set the MAC address within the VM.

As root, create a file /etc/udev/rules.d/75-mac-vmnet.rules with the following content:

ACTION=="add", SUBSYSTEM=="net", KERNEL=="enp0s3", RUN+="/usr/bin/ip link set dev %k address 00:11:22:33:44:55"
replace enp0s3 with the name of your interface and 00:11:22:33:44:55 with the desired MAC address.

Reboot or issue a ip link set dev enp0s3 address 00:11:22:33:44:55 to change your MAC address.

Reference: macos-vmnet

Currently semu does not support graphics rendering.

I think VirtIO GPU with 2D mode might be a good starting point to go.
Later we can then emulate mouse and keyboard for more sophisticated features.

Beginning with version 3.8 of Linux, users without privileged access have been able to create network_namespaces and user_namespaces. Despite this, the utility of unprivileged network namespaces was limited because the creation of virtual ethernet (veth) pairs, which bridge the host and network namespaces, still demanded root access, effectively barring internet connectivity.

The slirp4netns provides a solution to this limitation by enabling an unprivileged connection of a network namespace to the internet. It achieves this by linking a TAP device within a network namespace to a TCP/IP stack operating in user mode, known as slirp, thereby facilitating internet access without the need for root privileges. This means that it is feasible for users to integrate slirp4netns to run a RISC-V/Linux guest with full user-mode networking.