tyler569 / nightingale Goto Github PK

View Code? Open in Web Editor NEW

125.0 13.0 12.0 9.57 MB

A small operating system where I experiment and learn osdev.

License: GNU General Public License v3.0

Assembly 1.51% GDB 0.03% C 92.87% Shell 0.36% Ruby 3.89% Brainfuck 0.02% CMake 1.30% Makefile 0.01%

kernel osdev operating-system nightingale os

nightingale's Introduction

Nightingale

About

Nightingale is an operating system for x86_64 that I have been developing for 7 years to learn about low-level programming and operating system design.

Nightingale implements a mostly POSIX-like userland, though compliance is not a goal. I see POSIX as useful as a well-understood and documented interface, and one that permits compatability with large amounts of existing software.

For more specific feature and capability information, see ABOUT.md.

Building nightingale

Nightingale uses the CMake build system and defaults to the clang compiler.

The only uncommon package you will need is the grub-mkrescue tool provided by grub2. This is usually packaged with grub2 or in a 'grub tools' package.

Execute ./make in the root of the project.
To run, use ./run - its help text will show the available options

Project map

Directories

doc: Documentation
include: Header files for the whole system
interface: Interface definitions for syscalls and errno values
kernel: The core of the operating system
libc: Common userland routines, including things like printf
linker: Kernel module loader, userland dynamic linker, and libelf
script: Utility scripts for building and developing nightingale
user: In-tree usermode programs distributed with the system

Scripts

dump.rb: convenience wrapper around objdump
run.rb: convenience wrapper around qemu-system-x86_64 to set the options I need

Interface Manifests

These manifest files define the public syscall interface of the nightingale kernel, they are rendered into C enums and metadata that is used by both the kernel and the C library.

ERRNOS: defines the values of errno, their names, and their perror strings
SYSCALLS: defines syscall numbers, types, and arguments

nightingale's People

Contributors

Stargazers

Watchers

Forkers

felix0080 stackdoubleflow dalalsunil1986 carlosjordandev gnudimarik 00mjk mewbak bunnybunbun37204 birdiey xianlongliang kazi-code nomadarchitect

nightingale's Issues

Audit usage of integer types

Generally the core kernel shouldn't care about the exact bit width of integers - obvious exceptions to that rule are

arch/
net/
<drivers>

Most of the other uintXY_t types in the core kernel should (probably?) be replaced with core C types (int, long, short, etc) to make porting go more smoothly.

Crash in syscall should trace back through user code to main.

Crashes that occur inside a syscall should be able to rewind the stack that led to the invocation of the syscall and continue printing symols through userspace to improve the debugging experience.

Proper user environment / ENVP

A lot of places that I should support both ARGV and ENVP are just cop-outs, including the fact that ARGV currently uses the location called USER_ENVP to store its argument data.

User mode needs proper environment variable support.

`cat .` jumps to a null pointer

Nightingale shell
$ cat .
Fault reading instruction:0x0 because page not present from kernel mode.
NULL pointer access?
Fault occured at 0x0
    rax: fffffff000005150    r8 : ffffffffffffffff
    rbx: fffffff000000170    r9 :                0
    rcx: fffffff000004b10    r10: ffffffff8010ce2f
    rdx:               80    r11:                0
    rsp: fffffff00141ee40    r12: fffffff000005150
    rbp: fffffff00141ee68    r13:     7ffffeffff38
    rsi:     7ffffeffff38    r14:               80
    rdi: fffffff000005150    r15:     7ffffeffff38
    rip:                0    rfl: [      I       ] (202)
    cr3:           501000    pid:                3
backtrace from: 0xfffffff00141ee68
(0x0) <no match>
(0xffffffff80108a32) <do_syscall_with_table+0xf8>
(0xffffffff8010f34c) <syscall_handler+0x9c>
(0xffffffff8010f880) <c_interrupt_shim+0x7e>
(0xffffffff80107030) <interrupt_shim+0x30>
end of memory

Move kernel/*.h to kernel/include

The char_devices.h and membuf.h files are super weird just hanging out in kernel, they should be somewhere in include.

The open question is whether they deserve a kernel/include/ng/fs folder or not.
Either way will require #include massaging, which I don't want to do now, hence the issue.

sh doesn't like redirected shebangs

$ cat test.sh
#!/bin/sh

echo "Hello World"

$ #!/bin/sh
$ sh < test.sh
Unrecognized token-> 1

Trouble compiling

Sorry if I am missing something but I ran the toolchain script in a place that was in path. The compiler runs perfectly but when I run make it returns:

make -j 8 -C . INNER=1 ngos.iso
make[1]: Entering directory '/home/cayden/Documents/Code/Kernel/nightingale'
CC      syscall.o
CC      errno.o
CC      vector.o
CC      syscalls.o
CC      stdlib.o
CC      locale.ox86_64-nightingale-gcc: fatal error: cannot execute 'cc1': execvp: No such file or directory

compilation terminated.
x86_64-nightingale-gcc: fatal error: cannot execute 'cc1': execvp: No such file or directory
compilation terminated.
make[1]: *** [makefile.inc:18: build-x86_64/libc/syscall.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [makefile.inc:23: build-x86_64/libc/errno.o] Error 1
x86_64-nightingale-gcc: fatal error: cannot execute 'cc1': execvp: No such file or directory
compilation terminated.
make[1]: *** [makefile.inc:28: build-x86_64/libc/vector.o] Error 1
x86_64-nightingale-gcc: fatal error: cannot execute 'cc1': execvp: No such file or directory
compilation terminated.
make[1]: *** [makefile.inc:33: build-x86_64/libc/syscalls.o] Error 1
x86_64-nightingale-gcc: fatal error: cannot execute 'cc1': execvp: No such file or directory
compilation terminated.
make[1]: *** [makefile.inc:38: build-x86_64/libc/stdlib.o] Error 1
x86_64-nightingale-gcc: fatal error: cannot execute 'cc1': execvp: No such file or directory
compilation terminated.
make[1]: *** [makefile.inc:43: build-x86_64/libc/locale.o] Error 1
make[1]: Leaving directory '/home/cayden/Documents/Code/Kernel/nightingale'
make: *** [makefile:2: all] Error 2

What am I missing?
Thank you in advance.

Traceback when loading a module that is not in virtual memory causes exception

This case is invalid and should crash, but it should crash the thread, not the system.
0xffffffff8011811d is in elf_parse.

$ insmod testmod.ko
Thread: [6:6] ("insmod") performed an access violation
Fault reading data:0xffffffff8044e010 because page not present from kernel mode.
Fault occurred at 0xffffffff80118050
    rax:                0    r8 : ffffffff801f35b0
    rbx:                0    r9 : ffffffff801f3a70
    rcx:                b    r10:                0
    rdx: ffffffff801f286e    r11:                a
    rsp: ffffffffc165fda0    r12: ffffffff8044e010
    rbp: ffffffffc165fdc0    r13: ffffffff801f1980
    rsi: ffffffff8087cdcf    r14: ffffffff8087cdcf
    rdi: ffffffff801e4bc8    r15: ffffffff801e4bc8
    rip: ffffffff80118050    rfl: [ P Z          ] (46)
    cr3:            23000    pid: [6:6]
backtrace from: 0xffffffffc165fdc0
Thread: [6:6] ("insmod") performed an access violation
Fault reading data:0xffffffff8044e010 because page not present from kernel mode.
Fault occurred at 0xffffffff8011811d
    rax:                0    r8 : ffffffff8044e010
    rbx: ffffffff80118050    r9 :                1
    rcx:                0    r10:             1158
    rdx: ffffffff8044e028    r11:                0
    rsp: ffffffffc165fbf0    r12: ffffffffc165fdc0
    rbp: ffffffffc165fbf0    r13: ffffffff801b733e
    rsi: ffffffff80118050    r14:                0
    rdi: ffffffff801e4bc8    r15:                0
    rip: ffffffff8011811d    rfl: [ P Z          ] (46)
    cr3:            23000    pid: [6:6]
backtrace from: 0xffffffffc165fbf0

Break libc out into a project of its own

Motivations:

my Libc should be able to run on top of linux with minor modifications and that should allow me to perform much more robust testing of the logic using existing testing frameworks (the musl test suite for example)
this also provides a better way to test functionality of my user programs, as well as making their development easier without needing to rely on both qemu and a buggy kernel.

Fix sys_wait4()

I just ripped out wait4() since it was using a very nonstandard interface for that syscall. I can have the wait(pid) interface, but it shouldn't be called wait4.

wait4 should conform to the actual interface:

       wait3, wait4 - wait for process to change state, BSD style

SYNOPSIS         top

       #include <sys/types.h>
       #include <sys/time.h>
       #include <sys/resource.h>
       #include <sys/wait.h>

       pid_t wait3(int *wstatus, int options,
                   struct rusage *rusage);

       pid_t wait4(pid_t pid, int *wstatus, int options,
                   struct rusage *rusage);

Port to aarch64

As part of the portability initiative, I should port the OS to aarch64.

Move toward a more standard interface

Move away from custom types and custom headers toward the standard C ones - this is to provide maximum comprehensibility and interoperability going forward.

Block process without busy loop

During a blocking syscall, a process should be able to register with the scheduler that it does not want to be scheduled until the syscall can be completed.

This can potentially be interrupted by signals when they are implemented.

Rewrite malloc

The current implementation of malloc() for the kernel is a disaster, having caused more headaches and bugs than most of the rest of the system combined. It should be redone from the ground up in a more standard way.

Support thread-local storage

could Nightingale OS fit on a floppy?

Even today the floppies are still being used, for example - as virtual floppies inside the coreboot open source BIOS. Just imagine: your wonderful OS could be a part of someone's BIOS build! (for coreboot supported motherboard, maybe you have or could get one - see https://www.coreboot.org/Supported_Motherboards )

@tyler569 , If you already have a coreboot-supported motherboard, or a real chance to get one, - wouldn't it be cool to be able to launch your own OS straight from the BIOS chip? ;) With one simple command its possible to add any floppy to coreboot BIOS build - and then you see it as a boot entry! Multiple floppies could be added this way (as long as you have enough space left inside the BIOS flash chip, luckily LZMA compression could be used for the stored floppies to reduce their occupied size)

It should be possible to dump stack of a suspended process

This shouldn't be a large change, I imagine it should be as simple as calling backtrace_from on the saved base pointer and IP in the saved frame of the suspended task.

TCP

The network stack should support TCP.

This can start pretty basic doing lock-step ACKs, but should expand to include windowing and PSH OOB data.

Sockets need to be extended to support this, along with the relevant syscalls.

SIGSEGV does not work on x86_32

********************************

The Nightingale Operating System
Version v0.8.8-7-g2db9da8

********************************

pit: actual divisor: 1193
pit: ticking
cpu: allowing irqs
initialization took: 25036446
Hello World from a kernel thread
Nightingale shell
$ segv
Thread: [3:2] ("segv") performed an access violation
** Segmentation fault **
Attempted to access: data:0x0, Got: page not present
Fault occured at 0x80480ee
    eax:        0    ebx:        0
    ecx: 7fefffd4    edx:       29
    esi:        0    edi:        0
    esp: 7fefffc0    ebp: 7fefffc8
    eip:  80480ee    efl: [ P   I] (206)
    cr3:   6c3000    pid:        3
backtrace
    bp:         7fefffc8    ip:          80480ee
    bp:         7fefffc8    ip:          8048122
    bp:         7fefffec    ip:                0
Thread: [3:2] ("segv") performed an access violation
Fault writing data:0x2f9 because page not present from kernel mode.
NULL pointer access?
Fault occured at 0x80109207
    eax:      2fd    ebx: c0004960
    ecx: c0836acc    edx:      3fd
    esi: c0003ca0    edi: c0836fc0
    esp: c0836e78 ?  ebp: c0836f34
    eip: 80109207    efl: [ P  SI] (286)
    cr3:   6c3000    pid:        3
backtrace from: 0xc0836f34
(0x80109207) <do_signal_call+0x81>
(0x80109330) <send_immediate_signal_to_self+0x49>
(0x8010c29f) <page_fault+0x21e>
(0x8010c42c) <c_interrupt_shim+0x65>
(0x80104019) <interrupt_shim+0x19>
end of memory

It dies in do_signal_call setting up the return stack

[email protected]:214
[email protected]:8
[email protected]:382
[email protected]:212
return_from_interrupt@??:?