mit-pdos / biscuit Goto Github PK

Biscuit research OS

License: BSD 3-Clause "New" or "Revised" License

GDB 0.01% Makefile 0.02% Shell 0.15% Go 88.00% Assembly 4.37% C 1.56% Python 0.14% C++ 0.24% Fortran 0.01% HTML 5.41% JavaScript 0.01% Batchfile 0.02% Awk 0.01% Perl 0.08%

biscuit's Introduction

Biscuit research OS

Biscuit is a monolithic, POSIX-subset operating system kernel in Go for x86-64 CPUs. It was written to study the performance trade-offs of using a high-level language with garbage collection to implement a kernel with a common style of architecture. You can find research papers about Biscuit here: https://pdos.csail.mit.edu/projects/biscuit.html

Biscuit has some important features for getting good application performance:

Multicore
Kernel-supported threads
Journaled FS with concurrent, deferred, and group commit
Virtual memory for copy-on-write and lazily mapped anonymous/file pages
TCP/IP stack
AHCI SATA disk driver
Intel 10Gb NIC driver

Biscuit also includes a bootloader, a partial libc ("litc"), and some user space programs, though we could have used GRUB or existing libc implementations, like musl.

This repo is a fork of the Go repo (https://github.com/golang/go). Nearly all of Biscuit's code is in biscuit/.

Install

The root of the repository contains the Go 1.10.1 tools/runtime. Some of Biscuit's code is modifications to the runtime, mostly in src/runtime/os_linux.go.

Biscuit used to build on Linux and OpenBSD, but probably only builds on Linux currently. You must build Biscuit's modified Go runtime before building Biscuit:

$ git clone https://github.com/mit-pdos/biscuit.git
$ cd biscuit/src
$ ./make.bash

then go to Biscuit's main part and launch it:

$ cd ../biscuit
$ make qemu CPUS=2

Biscuit should boot, then you can type a command:

# ls

Troubleshooting

You need qemu-system-x86_64 and python2 in your environment. If your distribution does not name them that way, you have to fix the naming, path, etc.
If the GOPATH environment variable doesn't contain biscuit/, the build will fail with something like:

src/ahci/ahci.go:8:8: cannot find package "container/list" in any of:
...

Either unset GOPATH or set it explicitly, for example (assuming that your working directory is where the GNUMakefile is):

$ GOPATH=$(pwd) make qemu CPUS=2

Contributing

Please feel free to hack on Biscuit! We're happy to accept contributions.

biscuit's People

Contributors

Stargazers

Watchers

Forkers

sudarsunkannan iot-alex mikalv longjohncoder mewbak devhttps denisse-dev scoir shaunstanislauslau nonerkao resouer hthappiness forging2012 shammishailaj kybin crazyguitar 0x6b386f linecode takeharukato bfrancojr potatogim jafffy oksbsb evanphx ubuntu-repo mylinyuzhi joncampbell123 wachino canhunan1 s-you keyhanjk denismikhalev valery-barysok junftnt gzuhlwang dongzhaoyu tslilyai gmacd papaidas15790 fuath hamedcse kiwiflow gensfl16 biswapanda moneytech dolanor-galaxy jiegec woowa-kyunam wushilong pjcdarker bingwenwuhen victorcheng2333 mikasaakerman crixalis2013 ramla-i wenjun93 xxf09th jyizheng leezhenghui lxplore venmotools shizonic gocodeman thetrident backwardn funron whakapapa soomtong bonedaddy huaiguoguo fengpf zhangzhishuo beoran ops-l edwindvinas buffernihility zoonoo spacefan zhenxian-hu dermonarch nirvanagit gigimushroom hzmsrv at15 liweiyuan kunaldawn tommyliu1114 isgasho wuxyintel thurday jack-ji catsalt jeffcarl67 nikolaevarius hetong07 openexw tekktomato foxxnuaa jpillora hasheddan

biscuit's Issues

Got "64bit addressess not supported" when "make qemu CPUS=2"

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

go1.14.7 linux/amd64

Does this issue reproduce with the latest release?

Yes, I also tried go1.15, which has the same problem

What operating system and processor architecture are you using (`go env`)?

Ubuntu 16.04.7 LTS, Linux kernel 4.15.0-112-generic

GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

make qemu CPUS=2

What did you expect to see?

get into the Biscuit OS and be able to type "ls"

What did you see instead?

          BiscuitOS
      go version: go1.10.1

4763 MB of physical memory
CPUID: family: 6, model: 1a
sse3 true, ssse3 true, sse41 true, sse42 true, avx false
invariant TSC not supported
bmi1 false, bmi2 false
1GB pages not supported
No hardware performance monitoring
panic: 64bit addressess not supported

goroutine 1 [running]:
ahci.attach_ahci(0x8086, 0x2922, 0x2000)
/home/zym/src/biscuit/biscuit/src/ahci/ahci.go:94 +0x73a
pci.pci_attach(0x8086, 0x2922, 0x0, 0x4, 0x0)
/home/zym/src/biscuit/biscuit/src/pci/pci.go:221 +0xd2
pci.Pcibus_attach.func2(0x0, 0x4)
/home/zym/src/biscuit/biscuit/src/pci/pci.go:164 +0x9d
pci.Pcibus_attach()
/home/zym/src/biscuit/biscuit/src/pci/pci.go:179 +0x53
main.attach_devs(0x5a98e0)
/home/zym/src/biscuit/biscuit/src/kernel/main.go:952 +0x105
main.main()
/home/zym/src/biscuit/biscuit/src/kernel/main.go:1654 +0x1f7
exit with code 0000000000000002 .
halting

how build the project？

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

go version go1.16.6 linux/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (`go env`)?

x86-64 WSL2 Ubuntu

What did you do?

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

What did you expect to see?

What did you see instead?

`syscall` instruction unsupported

The kernel throws #UD (illegal instruction) error when syscall is ran; can it be enabled in boot?

Memory Allocation/Free unexpected behavior

What version of Go are you using (go version)?

Building with go version go1.6.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

I am building go.img with docker (Linux 16.04) but running QEMU on macOS Mojave with MacBook Pro 15' 2017.

What did you do?

I run QEMU with

make qemu CPUS=1
While I was allocating memory through malloc with the following C program

int* ptr = (int*)malloc(sizeof(int) * (1 << (18 + 8))); // allocating memory of size 256MB
free(ptr);

and repeatedly free and reallocate with following C program

for (int j = 0; j < 5; ++j) {
    for (int i = 4; i < 17; ++i) {
        int* ptr = (int*)malloc(sizeof(int) * (1 << (i + 8)));
        free(ptr);
        printf("%d %d\n", j, i);
    }
}

What did you expect to see?

I expected to see no free error which means that I am able to allocate memory and free successfully.

What did you see instead?

For allocating large amount of memory say 256MB, it fails. However when biscuit boots, it is boot with showing that we still have around 500MB free memory.

As for the second issue, it seems that free is not working successfully therefore I am getting out of memory error (malloc returns nullptr) after I have malloced with accumulated memory of size also around 256MB (about 128MB in total for first iteration and almost 128MB for second iteration then failed at the last malloc)

Thanks!

How to build Redis to run in biscuit?

It seems that biscuit has its own libc implementation and is able to run Redis as a user space program. But when I try to build redis with litc like this:

make CFLAGS="-fno-builtin -nostdinc -I/path/to/biscuit/biscuit/user/c/include -static" MALLOC=libc

It fails with the following errors:

anet.c: In function ‘anetV6Only’:
anet.c:420:35: error: ‘IPV6_V6ONLY’ undeclared (first use in this function)
     if (setsockopt(s,IPPROTO_IPV6,IPV6_V6ONLY,&yes,sizeof(yes)) == -1) {
                                   ^~~~~~~~~~~
anet.c:420:35: note: each undeclared identifier is reported only once for each function it appears in
anet.c: In function ‘_anetTcpServer’:
anet.c:434:13: error: ‘hints’ undeclared (first use in this function); did you mean ‘uint’?
     memset(&hints,0,sizeof(hints));
             ^~~~~
             uint
anet.c:437:22: error: ‘AI_PASSIVE’ undeclared (first use in this function)
     hints.ai_flags = AI_PASSIVE;    /* No effect if bindaddr != NULL */
                      ^~~~~~~~~~
anet.c:439:50: error: ‘servinfo’ undeclared (first use in this function); did you mean ‘sys_info’?
     if ((rv = getaddrinfo(bindaddr,_port,&hints,&servinfo)) != 0) {
                                                  ^~~~~~~~
                                                  sys_info
anet.c:443:10: error: ‘p’ undeclared (first use in this function)
     for (p = servinfo; p != NULL; p = p->ai_next) {

I tried to fix some of these, but more compilation errors appear afterwards. Could you please provide instructions on building Redis?

Unexpected runtime output

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

Building with go version go1.6.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (`go env`)?

I am building go.img with docker (Linux 16.04) but running QEMU on macOS Mojave with MacBook Pro 15' 2017.

What did you do?

I run QEMU with

make qemu CPUS=1

While I was benchmarking the process creating overhead with the following C program

#include "litc.h"
#include <unistd.h>

double create_process_overhead(int times){
    for (int i = 0; i < times; i++) {
        int pid = fork();
        if (pid == 0) exit(0);
        wait(0);
    }
    return 0;
}

int main(void){   
    for (int i = 0; i < 104; ++i) {
        create_process_overhead(500);
    }
    return 0;
}

What did you expect to see?

I expected to see no outputs.

What did you see instead?

I saw outputs of "POOF" randomly. And as long as I lower the times like calling creating_process_overhead(100), I haven't seen any "POOF" so far.

The output only exists when performing forking or pthread_create with large amount of times say 500.
Is it output by Biscuit? But I don't see any possible "POOF" string in the biscuit repository.
Thanks!

Userland binaries - How was redis and nginx compiled?

Hi, I am trying to port some userland applications to biscuit. How did you compile and test redis and nginx?

Also, why is USERMIN so high? Why not something like 0x400000 which makes it closer to compatible with Linux static binaries?

Fails to build if GOPATH is set in parent shell

biscuit/GNUmakefile has GOPATH ?= $(shell pwd) which only sets GOPATH if it's not already set. If the user has GOPATH set to any value that doesn't include the current directory as an element, the build fails:

~/src/biscuit/biscuit$ rm -f mkfs; GOPATH=/whatever make qemu CPUS=2
GOPATH="/whatever" ../bin/go build src/mkfs/mkfs.go
src/mkfs/mkfs.go:8:8: cannot find package "fs" in any of:
	/home/tv/src/biscuit/src/fs (from $GOROOT)
	/whatever/src/fs (from $GOPATH)
src/mkfs/mkfs.go:9:8: cannot find package "ufs" in any of:
	/home/tv/src/biscuit/src/ufs (from $GOROOT)
	/whatever/src/ufs (from $GOPATH)
src/mkfs/mkfs.go:10:8: cannot find package "ustr" in any of:
	/home/tv/src/biscuit/src/ustr (from $GOROOT)
	/whatever/src/ustr (from $GOPATH)
GNUmakefile:119: recipe for target 'mkfs' failed
make: *** [mkfs] Error 1

boot error

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

go version go1.13.8 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (`go env`)?

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jack/.cache/go-build"
GOENV="/home/jack/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jack/go"
GOPRIVATE=""
GOPROXY="https://goproxy.cn"
GOROOT="/usr/lib/go-1.13"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.13/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build706439729=/tmp/go-build -gno-record-gcc-switches"

What did you do?

following instructions of README.md mostly:

cd into src and build golang toolkit
cd into biscuit, adjusted GOPATH, QOPTS and run make qemu CPUS=2

Here's detailed modifications:

diff --git a/biscuit/GNUmakefile b/biscuit/GNUmakefile
index bb4cae8898..15b53ace54 100644
--- a/biscuit/GNUmakefile
+++ b/biscuit/GNUmakefile
@@ -8,7 +8,7 @@ ASFLAGS := $(BASEFLAGS) -nostdlib -nostdinc -I$(TOP)
 CFLAGS := $(BASEFLAGS) -ffreestanding -nostdlib -nostdinc -fno-builtin \
        -mno-red-zone -fno-stack-protector
 CXXFLAGS := $(BASEFLAGS) -ffreestanding -nostdlib -fno-builtin -mno-red-zone
-GOPATH ?= $(shell pwd)
+GOPATH := $(shell pwd)
 # bootloader sources
 ASMS := boot.S
 CS   := bootmain.c main.c chentry.c
@@ -85,7 +85,7 @@ SKELDEPS := $(shell find $(SKEL))
 
 CPUS := $(shell echo $${CPUS:-1})
 
-QOPTS := -m 8G -smp cpus=$(CPUS) -cpu Nehalem
+QOPTS := -m 2G -smp cpus=$(CPUS) -cpu Nehalem
 # for testing bhw2's configuration
 #QOPTS := -m 8G -cpu Nehalem -smp sockets=2,cores=10,threads=2

What did you expect to see?

kernel should boot without error

What did you see instead?

SeaBIOS (version 1.13.0-1ubuntu1)


iPXE (http://ipxe.org) 00:03.0 CA00 PCI2.10 PnP PMM+7FF8C9C0+7FECC9C0 CA00
                                                                               


Booting from Hard Disk...
 LAPIC Mhz: 00000000000003E7 
 CPU Mhz: 0000000000000AF8 
 no random!
Reserved 65536 pages (256MB)
              BiscuitOS
          go version: go1.10.1
  1685 MB of physical memory
CPUID: family: 6, model: 1a
sse3 true, ssse3 true, sse41 true, sse42 true, avx false
invariant TSC not supported
bmi1 false, bmi2 false
1GB pages not supported
No hardware performance monitoring
AHCI 8086 2922 (0:4:0), bara 0xfebb1000, MSI 57
ACPI CPUs 2, found 1 APs (0 hyperthreads) across 2 packages
Joining at most 4 CPUs split between packages (no hyperthreads) on 2 packages
 cpu 0000000000000001 joined
done! 1 APs found (1 joined)
Package counts:
  0:   1
  1:   1
Using MEMORY FS
FS log length 1024, maxtrans 512
no FS recovery needed: head 82
start [bin/init []]
read too few elf bytes (program headers)
panic: exec failed -1

goroutine 1 [running]:
main.main.func2(0xc000512260, 0x8, 0x8, 0x0, 0x0, 0x0)
	/home/jack/osdev/biscuit/biscuit/src/kernel/main.go:1688 +0x668
main.main()
	/home/jack/osdev/biscuit/biscuit/src/kernel/main.go:1693 +0x38a
 exit with code 0000000000000002 .
halting

how can I exit from biscuit?

Thank you for building this. I am also very interested in OSes made by go.

I've look around in it. It was fun.

But I don't find the exit command. Need a help.

Write User-space Go for biscuit

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (`go version`)?

go version go1.10.3 linux/amd64

Does this issue reproduce with the latest release?

N/A

What operating system and processor architecture are you using (`go env`)?

GOOS="linux"
GOARCH="amd64"

What did you do?

N/A

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

What did you expect to see?

That I am able to run Go binaries in biscuit

What did you see instead?

N/A

Are there any support in biscuit to run user-space Go binaries? I see C++ and C (e.g. user/c/lsh.c) but no Go. Where do I start?

Networking

I wonder how do we add interfaces or how do we modify biscuit to let it be able to connect to the internet. When I tried the command lnc to connect to my custom host outside QEMU, it has the message saying no route to host. After I dig into the kernel, I noticed that it only has 127.0.0.1 and a magic ip and gateway defined in src/ixgbe/ixgbe.go as follow:

// 18.26.5.49 (bhw)
me := Ip4_t(0x121a0531)
x.ip = me
bnet.Nic_insert(me, x)

netmask := Ip4_t(0xfffffe00)
// 18.26.5.1
gw := Ip4_t(0x121a0401)

I wonder if I am correct that this is the place where biscuit setup its network interface and how the ip and gateway is selected. In addition, how we should configure to make biscuit able to connect to hosts outside QEMU. Also, does biscuit provide ping and ICMP packet processing? Thanks.

mit-pdos / biscuit Goto Github PK

biscuit's Introduction

Biscuit research OS

Install

Troubleshooting

Contributing

biscuit's People

Contributors

Stargazers

Watchers

Forkers

biscuit's Issues

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

Recommend Projects

Recommend Topics

Recommend Org

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?