returninfinity / baremetal Goto Github PK

For running the kernel in VMware.

Add API functions to assign work to each CPU core individually.

Not as simple as the work queue but will be handy once NUMA comes into play.

Todo:

• Remove system calls b_smp_enqueue, b_smp_dequeue, b_smp_queuelen, b_smp_wait
• Remove system variables os_QueueLen, cpuqueue, cpuqueuestart, cpuqueuefinish, cpuqueuemax, cpustatus
• Add a 4K table for CPU jobs where each entry has a code and var address (16 bytes each).
• Add a system call call b_smp_set that take 3 variables - CPU ID, code address, data address
• b_smp_set checks the entry to see if it is currently NULL, if so add the values, if not bail out
• b_smp_set also sends a wakeup signal to that CPU so it can check the work table.

The CPU idle loop will check the work table, execute the code at the address specified, and clear its entry in the work table when the code has completed execution.

Set all table records to 0xFFFFFFFFFFFFFFFF initially. CPU init/reset will have each core set it to 0.

The kernel still calls the built-in input/output functions if something (like Alloy) changes the kernel function table. The kernel should use the public function table and not internal addresses for the default functions.

Only the first char is outputted. Hindering debugging.

Pure64 did a few extra things while in 16-bit mode via BIOS interrupts. Since GRUB starts in 32-bit mode we can't use those interrupts. The memory map is handled by GRUB so no worries there.

; Configure serial port
xor dx, dx ; First serial port
mov ax, 0000000011100011b ; 9600 baud, no parity, 1 stop bit, 8 data bits
int 0x14

; Make sure the screen is set to 80x25 color text mode
mov ax, 0x0003 ; Set to normal (80x25 text) video mode
int 0x10

; Disable blinking
mov ax, 0x1003
mov bx, 0x0000
int 0x10

What happened to video mode in the kernel?

Why did it get removed?

First off, sorry this bug report isn't too specific about a number of things, mostly involving steps to reproduce the problem... Anyway, the problem is that BM hangs a lot. I have been playing with modifying the c source for the hello world demo app, by doing simple things like adding another line of output using b_output(), and it will often hang up and not go back to the command prompt. Other times it will print both lines correctly, but adding any more code after that and it starts hanging again.

I've tried recompilation without any code changes, and it has acted differently both times! Sometimes there is a message about an interrupt being handled, other times it just silently crashes. I've tried building my own qemu binary from current git sources, but the problem persists. The BM tree is from current git, too....

Go ahead and close this bug if you are unable to reproduce it, and I'll chalk it up to not wearing my tinfoil hat or something :-). I just wanted to vent. And please let me know if there is a good way to track down the problem, too - maybe I could get someone some debug logs or something.

I noticed that newlib is currently listed as something that BareMetal-kernel works with.

Does it really?

I would like to test BareMetal-kernel.
I have compiled kernel.sys with build_x86-64.sh script.
In src/X86-64 readme i can see that we use boot.bin file with qemu but i don't understand how to create boot.bin from kernel.sys.
Can you explain more in depth how to compile and run BareMetal-kernel with qemu ?

• Separate the keyboard and RTC drivers into their own files (instead of just sitting in interrupt.asm)
• Move screen functions to a new console driver

• Add a video driver so we can display characters in non-text modes (needed for EFI systems)
• Use the work done in BareMetal OS

A fatal exception is currently sending incomplete data to the serial port. This needs to be corrected for easier debugging. Also create a detailed document on debugging the kernel.

Add Virtio network support for running BareMetal in some VMs.

http://wiki.libvirt.org/page/Virtio

PCI ID is 0x1AF4:0x1000 - http://pci-ids.ucw.cz/read/PC/1af4/1000

Specs: http://docs.oasis-open.org/virtio/virtio/v1.0/virtio-v1.0.pdf

Legacy specs: http://ozlabs.org/~rusty/virtio-spec/virtio-0.9.5.pdf

Some example C code:
http://dox.ipxe.org/virtio-net_8c_source.html

• AHCI driver must search for controllers via all possible PCI IDs and save BAR5 (AHCI Base Address Register) for each detected instead of just stopping at the first.
• Store information on if drives are detected on each controller. AHCI supports 32 devices via a controller as per the AHCI specs. Drive numbers will be assigned based on controller and port numbers. First port on first controller is drive 0. Last port on first controller is drive 31. First port on second controller is drive 32. And so on.
• Add call for drive info to return 512-byte identity data.
• Enough memory will need to be reserved for the structures needed.

The kernel uses a non-standard way of passing variables to internal functions. The AMD64 ABI should be investigated to see if it is a good fit.

From the AMD64 ABI:
Passing Once arguments are classified, the registers get assigned (in left-to-right order) for passing as follows:

1. If the class is MEMORY, pass the argument on the stack.
2. If the class is INTEGER, the next available register of the sequence %rdi,
   %rsi, %rdx, %rcx, %r8 and %r9 is used.
3. If the class is SSE, the next available vector register is used, the registers
   are taken in the order from %xmm0 to %xmm7.

Return values are passed as %rax, %rdx

Linux system call table

I'm wondering at what point is the armv8 architecture going to be supported, as I just found out that the Raspberry Pi 3 Model B uses this architecture. It would be interesting to have BareMetal run on a Pi.

I could start working on this, if you want. I've wanted to purchase a Pi for a while now, and I'm betting there's emulators for armv8 too that I could use to get it started.

How to reproduce:

Boot a host via PXE. Start pinging random IP address on the PXE boot network to generate ARP packets

Should export in the format:

Memory address (rounded down), Hex values
0x0123456789ABCDEF: 00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F

Double space between 07 and 08 to separate (hopefully) aligned 64-bit values.

Current memory usage defined by the kernel:

os_EthernetBuffer: equ 0x0000000000140000 ; 262144 bytes 0x140000 -> 0x17FFFF Location of Ethernet RX Ring Buffer - Room for 170 packets
os_ethernet_rx_buffer: equ 0x00000000001C0000
os_eth_rx_buffer: equ 0x00000000001C8000
os_ethernet_tx_buffer: equ 0x00000000001D0000
os_eth_tx_buffer: equ 0x00000000001D8000

• Clean up and minimize the memory usage
• Better descriptions of what memory is for what purpose

Add new public functions for dealing with PCI read/writes.

Add code to b_smp_config for checking if a CPU is present/active/idle.

RCX should be passed for an CPU ID.

• Rename functions to use ahci_
• Remove the check for the starting sector in read and write and let the hardware return the error.

• Remove b_output and b_input
• Rename b_output_chars to b_output
• Rename b_input_key to b_input

Memory on x86-64 is a bit of a mess. Unusable sections and gaps are a pain.

http://wiki.osdev.org/Detecting_Memory_(x86)

Map valid memory into a contiguous space in the higher half.

The kernel needs code for loading the middleware from disk/memory to the appropriate area and start it.

The documentation says that system functions are b_something and some even show examples for that, but in actual code it's os_.

First of all, the NVME code has been a great help for me to develop my own driver for my toy OS. It's well documented and well written. I did however find a few issues. I'll post them all here for brevity, but they do deserve to be their own issues if taken seriously.

Missing wait loop when disabling the controller during initialization

When disabling the controller, a loop needs to wait for it to be actually disabled before continuing the initialization. Without this, the next initialization steps (creating the admin queues) can fail. I can reproduce this issue on my test laptop (an Asus Vivobook)

Missing error checking when waiting for command completion

In both the admin and I/O completion checks, DW2 is checked to be non zero. In fact, the comments say "Add 8 for DW3", but yet 8 is added, which is the offset for DW2. This check will correctly identify that the command completed. However, it would be better if the status code in DW3 was checked to be zero.

LBA size determination is incorrect

The code finds the highest possible LBA size and assumes the drive was formatted with it. I believe the correct thing to do is read the FLBAS byte, then use that in the LBA format table LBAF table to look up the LBA size of the formatted drive.

Since it is not really part of the kernel.

• Commit multiboot changes to Pure64
• Create new boot sectors that switch to 32-bit mode
• Update documentation

Secure the kernel - Use 4K pages with read-only
Crashes in user code must be recoverable

The AHCI driver only uses 512B sectors. ID should be checked to see if support for larger sectors is supported.

Check bit 12 of word 106 first to see if its set. If so, check words 117-118.

From the ATA8 Command Set, IDENTIFY returns the following:

7.17.7.73 Word 106: Physical sector size / Logical Sector Size
If bit 14 of word 106 is set to one and bit 15 of word 106 is cleared to zero, the contents of word 106 contain valid information. If not, information is not valid in this word.
Bit 13 of word 106 shall be set to one to indicate that the device has more than one logical sector per physical sector.
Bit 12 of word 106 shall be set to 1 to indicate that the device has been formatted with a logical sector size larger than 256 words. Bit 12 of word 106 shall be cleared to 0 to indicate that words 117-118 are invalid and that the logical sector size is 256 words.
Bits (11:4) of word 106 are reserved.
Bits (3:0) of word 106 indicate the size of the device physical sectors in power of two logical sectors.
Examples:
Bits (3:0): 0 = 1 logical sector per physical sector
Bits (3:0): 1 = 2 logical sector per physical sector
Bits (3:0): 2 = 4 logical sector per physical sector
Bits (3:0): 3 = 8 logical sector per physical sector

7.17.7.78 Words 117-118: Logical Sector Size
Words 117,118 indicate the size of device logical sectors in words. The value of words 117,118 shall be equal to or greater than 256. The value in words 117,118 shall be valid when word 106 bit 12 is set to 1. All logical sectors on a device shall be 117,118 words long.

When trying to compile the kernel.asm file into kernel.sys I constantly get this error:

obi@ubuntu:~/Desktop/kernel/src/x86-64$ nasm kernel.asm -o ../../kernel.sys
sysvar.asm:69: error: symbol `os_net_ack_int' redefined

I am running Ubuntu 14.10 x64

http://en.wikipedia.org/wiki/Zero-copy

This is partially implemented.

b_net_tx forwards the memory location of the packet directly to the network driver and the hardware handles it. No time is spent by the CPU to copy anything.

b_net_rx does one copy (from the OS packet buffer to the requested memory address) via a nasty rep movsb so that will need to be adjusted in the network drivers.

On startup a default OS buffer location will be configured. Via a system call an application will be allowed to modify this to suit its own location.

minIP is here: https://github.com/IanSeyler/minIP

It's a small C app with a very basic IP stack and web server.

it seems in the code of ahci.asm

	mov eax, 0x00010005		; 1 PRDTL Entry, Command FIS Length = 20 bytes
	cmp ebx, 0x35			; Was a write requested?
	jne ahci_io_skip_writebit	; If not, skip setting the write flag
	bts ebx, 6			; Set the write flag (bit 6)
ahci_io_skip_writebit:
	stosd				; DW 0 - Description Information

There is a bug, the write flag bit 6 should be set for eax?

So recently I've been working on a sort of console/terminal for BareMetal-kernel, but for some reason, the disk doesn't seem to be reading.

I'm calling it like this:

    ; read manager app
    mov rax, 6
    mov rcx, 1
    mov rdx, 0
    mov rdi, 0x0000000000200000
    call os_disk_read

    ; call manager app
    mov rax, 0x0000000000200000
    call os_smp_enqueue

I know as a fact (from hexdump) that my program is on the disk at 24576 bytes (6 4096-byte sectors) in.

Is this a problem with my program, am I calling the function incorrectly from assembly, or is there some other reason why this isn't working?

Move the debugging calls to a new file.

Make sure network drivers push/pop the registers they use. Currently the b_net calls handle this.

Newlib is released under a variety of licenses, have you thought about using the Public Domain C library (pdclib) as your standard library?

Create a new file called config.asm for b_system_config to live in.

System configuration should all be done by that function. PCI read/writes, disk information, etc.