buaales / hagfish Goto Github PK

= Hagfish =

This is the Barrelfish/ARMv8 UEFI loader prototype: Hagfish (it's a basal
chordate i.e. something like the ancestor of all fishes).

== What Is Hagfish ==

Hagfish is a second-stage bootloader for Barrelfish on UEFI platforms, most
importantly the ARMv8 server platform.

Hagfish is loaded as a UEFI application, and uses the large set of supplied
services to do as much of the one-time (boot core) setup that the CPU driver
needs as is reasonably possible.  More specifically, Hagfish:
 * Is loaded over BOOTP/PXE.
 * Reuses the PXE environment to load a `menu.lst`-style configuration.
 * Loads the kernel image and the initial applications, according to configuration.
 * Allocates and builds the CPU driver's initial (direct-mapped) page table.
 * Activates the initial page table, and allocates a stack.

== Why Another Bootloader? ==

The ARMv8 machines that we're porting to are different both to existing ARM
boards, and to x86.  They have a full pre-boot environment, unlike most
embedded boards, but it's not a PC-style BIOS.  The ARM Server Base Boot
Requirements [1] specify UEFI.  Moreover, there is no mainline support from
GNU GRUB for the ARMv8 architecture, so no matter what, we need some amount of
fresh code.

Given that we had to write at least a shim loader, and keeping in mind that
UEFI is multi-platform (and becoming more and more common in the x86 world),
we're taking the opportunity to simplify the initial boot process within the
CPU driver by moving the once-only initialisation into the bootloader.  In
particular, while running under UEFI boot services, we have memory allocation
available for free, e.g. for the initial page tables.  By moving ELF loading
and relocation code into the bootloader, we can eliminate the need to relocate
running code, and can cut down (hopefully eliminate) special-case code for
booting the initial core.

== Technical Details ==

=== Assumptions and Requirements ===

Hagfish is (initially at least) intended to support development work on
AArch64 server-style hardware and, as such, makes the following assumptions:

 * 64-bit architecture, using ELF binaries.  Porting to 32-bit architectures
   wouldn't be hard, though.
 * PXE/BOOTP/TFTP available for booting.  Hagfish expects to load its
   configuration, and any binaries needed, using the same PXE context with
   which it was booted.  Changing this to boot from a local device (e.g. HDD)
   wouldn't be hard, as the LoadFile interface abstracts from the hardware.

=== Boot Process ===

In detail, Hagfish currently boots like this:

 1. Hagfish.efi is loaded over PXE by EFI, and is executed at a
    runtime-allocated address, with translation (MMU) and caching enabled.
 2. Hagfish queries EFI for the PXE protocol instance used to load it, and
    squirrels away the current network configuration.
 3. Hagfish loads the file `hagfish.A.B.C.D.cfg` from the TFTP server root
    (where A.B.C.D is the IP address on the interface that ran PXE).
 4. Hagfish parses its configuration, which is essentially a GRUB `menu.lst`,
    and loads the kernel image and any additional modules specified therein.
    All ELF images are loaded into page-aligned regions of type
    `EfiBarrelfishELFData`.
 5. Hagfish queries EFI for the system memory map, then allocates and
    initialises the initial page tables for the CPU driver (1-1 mapping of all
    occupied physical addresses).  The frames holding these tables are marked
    with the EFI memory type, `EfiBarrelfishBootPagetable`, allocated from the
    OS-specific range (`0x80000000-0x8fffffff`).  All memory allocated by
    Hagfish on behalf of the CPU driver is page-aligned, and tagged with an
    OS-specific type, to allow EFI and Hagfish regions to be safely reclaimed.
 6. Hagfish builds a Multiboot 2 information structure, containing as much
    information as it can get from EFI, including:
  * ACPI 1.0 and 2.0 tables.
  * The EFI memory map (including Hagfish's custom-tagged regions).
  * Network configuration (the saved DHCP ack packet).
  * The kernel command line.
  * All loaded modules.
  * The kernel's ELF section headers.
 7. Hagfish allocates a page-aligned kernel stack (type
    `EfiBarrelfishCPUDriverStack`), of the size specified in the configuration.
 8. Hagfish terminates EFI boot services (calls ExitBootServices), activates
    the CPU driver page table, switches to the kernel stack, and jumps into
    the relocated CPU driver image.

=== Post-boot state ===

When the CPU driver on the boot core begins executing, the following
statements hold:
 * The MMU is configured with a 1-1 translation of all RAM and I/O regions.
 * The CPU driver's code and data are both fully relocated into one or more
   distinct 4kiB-aligned regions.
 * The stack pointer is at the top of a distinct 4kiB-aligned region of at
   least the configured size.
 * The first argument register holds the Multiboot 2 magic value.
 * The second holds a pointer to a Multiboot 2 information structure, in a
   distinct 4kiB-aligned region.
 * The console device is configured.
 * Only one core is enabled.
 * The Multiboot structure contains at least:
  * The final EFI memory map, with all areas allocated by Hagfish to hold data
    passed to the CPU driver marked with OS-specific types, all of which refer
    to non-overlapping 4kiB-aligned regions:
       EfiBarrelfishCPUDriver ::
           The currently-executing CPU driver's text and data segments (these
           may be allocated together or separately).
       EfiBarrelfishCPUDriverStack ::
           The CPU driver's stack
       EfiBarrelfishMultibootData ::
           The Multiboot structure.
       EfiBarrelfishELFData ::
           The unrelocated ELF image for a boot-time module (including that
           for the CPU driver itself), as loaded over TFTP.
       EfiBarrelfishBootPageTable ::
           The currently-active page tables.
  * The CPU driver (kernel) command line.
  * A copy of the last DHCP ack packet.
  * A copy of the section headers from the CPU driver's ELF image.
  * Module descriptions for the CPU driver and all other boot modules.
 * If EFI provided an ACPI root table, the Multiboot structure contains a
   pointer to it.

== Using Hagfish ==

=== Downloading ===

The current Hagfish development tree is available at

```
  git://git.barrelfish.org/git/Hagfish
```

Clone it with:
```
    $ git clone git://git.barrelfish.org/git/Hagfish
```

=== Building ===

Hagfish is built using the UEFI embedded development kit (EDKII).  There are a
few bugs in the upstream EDK version that prevent Hagfish from building.  For
the moment, you'll need a patched version, available at

```
  git://git.barrelfish.org/git/uefi-edk
```

You'll also need an AArch64-compatible cross compiler, for example a recent
Linaro[2] toolchain (e.g. gcc-linaro-4.9-2014.11, installed under /opt in the
following example).

Once the cross-compiler is installed, build Hagfish as follows:
Set up a UEFI build environment:
```
    $ git clone git://git.barrelfish.org/git/uefi-edk
    $ cd uefi-edk
    $ make -C BaseTools/
    $ export EDK_TOOLS_PATH=$(pwd)/BaseTools
    $ . edksetup.sh BaseTools
    $ export CROSS_COMPILE=aarch64-linux-gnu-
```
Checkout/extract/copy Hagfish into a subdirectory of uefi-edk, then:
```
    $ build -a AARCH64 -t ARMGCC -p Hagfish/Hagfish.dsc -m Hagfish/Application/Hagfish/Hagfish.inf -b DEBUG
```
The image is at Build/Hagfish/DEBUG_ARMGCC/AARCH64/Hagfish.efi

=== Booting ===

Once you've got a copy of `Hagfish.efi`, copy it into the TFTP server's
subtree and configure your DHCP server to pass its relative path to the
machine you wish to boot.  The Mustang boards will automatically attempt a PXE
boot, as should most EFI machines, where no local boot device is configured.

=== Debugging ===

What you need:
 * GDB for AARCH64. If not in your OS's repository, go to [2]
 * Symbol file: Build/Hagfish/DEBUG_ARMGCC/AARCH64/Hagfish/Application/Hagfish/Hagfish/DEBUG/Hagfish.dll

Steps:

 * Set WAIT_FOR_GDB = 1 in Hagfish.dsc
 * rebuild Hagfish
 * extract the debug symbols:
  $ aarch64-linux-gnu-objcopy --only-keep-debug \
      Build/Hagfish/DEBUG_ARMGCC/AARCH64/Hagfish/Application/Hagfish/Hagfish/DEBUG/Hagfish.dll \
      Hagfish.sym
 * run GBD and attach to host, e.g. QEMU
 * start the execution until you see: Hagfish loaded at 78449000, size 310304B, by handle 7B0CCB98
 * (gdb) info files
	Entry point: 0x0
	0x0000000000000000 - 0x000000000003f924 is .text
	0x000000000003f928 - 0x0000000000046026 is .rodata
	0x0000000000056028 - 0x0000000000056e3a is .data
	0x0000000000056e40 - 0x000000000005ad90 is .bss
	0x0000000004ad0968 - 0x0000000004b1028c is .text in /local/code/hagfish.sym
	0x000000000003f928 - 0x0000000000046026 is .rodata in /local/code/hagfish.sym
	0x0000000000056028 - 0x0000000000056e3a is .data in /local/code/hagfish.sym
	0x0000000000056e40 - 0x000000000005ad90 is .bss in /local/code/hagfish.sym
 * Add the symbols at the right address: Use the load address and calculate the
   new location of the data section (0x56e3a + 0x78449000)
   (gdb) add-symbol-file Hagfish.sym 0x78449000 -s .data 0x784afe3a
 * (gdb) set variable wait = 0
 * (gdb) continue

=== Configuration ===

Hagfish configures itself by loading a file whose path is generated from its
assigned IP address.  Thus if your development machine receives the address
`192.168.1.100`, Hagfish will load the file `hagfish.192.168.1.100.cfg` from
the same TFTP server used to load it.  The format is intended to be as close
as practical to that of an old-style GRUB menu.lst file.  The following
example loads `/armv8/sbin/cpu_apm88xxxx` as the CPU driver, with arguments
`loglevel=4`, and an 8192B (2-page) stack.

--- Configuration example ---
#
# This script is used to describe the commands to start at
# boot-time and the arguments they should receive.
#

kernel /armv8/sbin/cpu_apm88xxxx loglevel=4
stack 8192
module /armv8/sbin/cpu_apm88xxxx
module /armv8/sbin/init

# Domains spawned by init
module /armv8/sbin/mem_serv
module /armv8/sbin/monitor

# Special boot time domains spawned by monitor
module /armv8/sbin/chips boot
module /armv8/sbin/ramfsd boot
module /armv8/sbin/skb boot
module /armv8/sbin/kaluga boot
module /armv8/sbin/spawnd boot bootarm=0
module /armv8/sbin/startd boot

# General user domains
module /armv8/sbin/serial auto portbase=2
module /armv8/sbin/fish nospawn
module /armv8/sbin/angler serial0.terminal xterm

module /armv8/sbin/memtest

module /armv8/sbin/corectrl auto
module /armv8/sbin/usb_manager auto
module /armv8/sbin/usb_keyboard auto
module /armv8/sbin/sdma auto

== Copyright ==

Most of the code in Hagfish is owned by ETH Zuerich, and released under the
license terms given in the LICENSE file.  If any files are under a different
license (or from a different owner), they are marked as such in their header.

== References ==
[1] http://infocenter.arm.com/help/topic/com.arm.doc.den0044a/Server_Base_Boot_Requirements.pdf
[2] https://www.linaro.org