Is there a good way to ask questions about the content? I don't think making an issue per question is ideal, but since this is a meta question about questions (and how they'd be handled in regards to the book) I think this one will be ok.

For instance, right now I want to ask a question about the multiboot_header.asm file, best place to do it?

The book clearly says a word is 16 bits below.
src/multiboot-headers.md:

Finally, the dd. It’s short for ‘define double word’. It declares that we’re going to stick some 32-bit data at this location. Remember, when x86 first started, it was a 16-bit architecture set. That meant that the amount of data that could be held in a CPU register (or one ‘word’ as it's commonly known) was 16 bits. To transition to a 32-bit architecture without losing backwards compatibility, x86 got the concept of a ‘double word’ or double 16 bits.

I am not sure following paragraph is true.
src/hello-world.md:

Okay, so we copied four hexadecimal digits into memory, right? For our H. 0248. A hexadecimal digit has sixteen values, so two of them are 32. Since we need one word for the colors, and one word for the H, that’s two words. Hence, if our first memory position is at 0, the second letter will start at 2.

http://wiki.osdev.org/Printing_To_Screen
Here on this page it says "Text mode memory takes two bytes for every "character" on screen. One is the ASCII code byte, the other the attribute byte."

And 0x0248 is two bytes( i.e. 16 bits), not two words(i.e. 32 bits).

I suspect we should say "...we need one byte for the colors, and one byte for the H, that is two bytes."
Memory position is increased by 2, not because that's two words, but two bytes.

Current text is
"Okay, so we copied four hexadecimal digits into memory, right? For our H. 0248. A hexadecimal digit has sixteen values, so two of them are 32. Since we need one word for the colors, and one word for the H, that’s two words. Hence, if our first memory position is at 0, the second letter will start at 2."

and this is definitely confusing. I suppose what you want to say is something like

"A hexadecimal digit has sixteen values and takes up four bits (since 2^4 = 16), so a sequence of two hexadecimal digits requires one 8-bit byte, and four hexadecimal digits take up two bytes. Hence, if our first memory position is at 0, the second letter will start at 2".

When building the .iso file, grub-mkrescue aborted with the error
xorriso : FAILURE : Cannot find path '/efi.img' in loaded ISO image

This from the command given in 3.3 Making an ISO.

Someone who writes Javascript all day, for example, might be a little unfamiliar with what is meant by the x86 architecture. We should probably include a small section discussing the ideas that there are different CPU architectures, that since we're working at such a low level we have to worry about such things and that we are targeting (at first) just Intel's x86 architecture.

Architectural differences have impacts on what we're doing (e.g. on the kind of assembly we are writing). We should make sure to highlight these differences.

Hi,

The explanation for this part:

; Point the first entry of the level 4 page table to the first entry in the
; p3 table
mov eax, p3_table
or eax, 0b11
mov dword [p4_table + 0], eax

Says that

Now, we defined p3_table in the BSS section, which means that it will start as all zeroes. So when we or with 0b11, it means that the first two bits will be set to one, keeping the rest as zeroes.

So here it seems like we moved the value of whatever was at p3_table there. However, it seems that it would make more logical sense if this was the address of the p3_table and later on the text does seem to confirm that this is what's going on:

We have to put the address of the level four page table in a special register

; move page table address to cr3
mov eax, p4_table

So I found this somewhat confusing because I wasn't sure if we were dealing with values or addresses.

Aside from that I noticed two other things in the page:

• I think it could also be helpful to clarify the bit structure of all 32 bits of the page table entry just to show which bits do what,
• In a few places, the ASM seems to operate on eax by default (i.e.: `mul ecx'). Might be good to note that.

Again this is pretty good work so far, really enjoying following this. :)

I think the OS X script may be incompatible with zsh, although I haven't quite pinned down exactly what about it is the problem. When I ran it using bash everything went smoothly, which leads me to believe it actually is some incompatibility between bash and zsh.

It would be great if the script could be updated to work with either of these, but I realize at some point there are plenty of shells people might want it to work with, so perhaps at least mentioning that it should be run with bash would suffice.

So, when enabling PAE:

    ; enable PAE
    mov eax, cr4
    or eax, 1 << 5
    mov cr4, eax

Why do we put the fifth bit to 1? What do other bits mean? It would be great to have some more info. The same happens when setting the long mode bit:

    ; set the long mode bit
    mov ecx, 0xC0000080
    rdmsr
    or eax, 1 << 8
    wrmsr

It says

This is just what you have to do to set this up. Again, we won’t get into too much detail, as it’s not very interesting.

But, well, it would at least be interesting to say what that 0xC0000080 means, or why is it there, and why is the 8th bit being set to 1.

And finally, when enabling paging:

    ; enable paging
    mov eax, cr0
    or eax, 1 << 31
    or eax, 1 << 16
    mov cr0, eax

It would be great to explain what those 31st and 16th bits do, and what other bits are there.

I'm wondering if it's worth having an aside on the concept of memory mapping. It's mentioned twice in Hello, world! without any explanation. If you think it's worthwhile, I'm happy to volunteer since this is 100% the kind of thing I like writing about in the voice of this book.

Just calling x86's hlt in a bootloader is likely not safe as it will continue when the first interrupt arrives assuming interrupts are on (and they will arrive a lot as the system timer interrupts a lot.. and so would the keyboard.. and any event the BIOS understands). It's much safer to disable interrupts first with cli.

Setting up the build on windows isn't actually that hard once you've got the right binaries. Getting the binaries however are the problem. I propose we get together a .zip package containing all the tools needed to start kernel developing. Here's a list of the tools we would need binaries of:

• Rust: Extremely easy to install and set up, doesn't have to be included
• qemu: Easy to install, perhaps we should include it anyways though. If not then at least drop a link to it on the windows specific page (edit: because it's not a simple self-contained tool that's easily installed, let's avoid including it)
• nasm.exe: Pre-compiled already available, I still think we should bundle it
• x86_64-elf-ld.exe: Should be just a compile of binutils away
• make.exe: Available through msys2, I still think we should bundle it
• grub-mkrescue: A nightmare to get it seems, I've heard rumors it builds under cygwin but couldn't confirm. If there's alternative ways to set up the image that would be great.
• xorriso: Windows alternatives exist but if we can just get xorriso it would be nice, I haven't looked much into this.

Which sweeps you through to the next chapter when you're only trying to click the scrollbar.

I couldn't find a low-impact solution, except giving it a right: 15px, which is kind of arbitrary and ugly.

It was introduced before https://github.com/intermezzOS/book/blame/master/src/hello-world.md#L172 but is not used at this location, leading to ambiguous sentences.

This probably got left behind in the midnight writing spree, but the makefile copies grub.cfg on the root level into build/isofiles/boot/grub, assuming that grub.cfg is at the aforementioned root level. But anyone following along would have grub.cfg still buried in isofiles/boot/grub.

I'm a bit confused about the header length value.

So we have this code:

header_start:
    dd 0xe85250d6                ; magic number
    dd 0                         ; protected mode code
    dd header_end - header_start ; header length

    ; checksum
    dd 0x100000000 - (0xe85250d6 + 0 + (header_end - header_start))
header_end:

dd 0xe85250d6 is hardcoded
dd 0 is hardcoded (and why is this in decimal and not hex, even though 0 is 0 regardless of number system)?

but line 3 is a size, but size of what exactly? Is the amount of memory the magic number and protected mode code take up? is it the size of the total instructions in the file? I feel like this is a core idea of assembly I'm missing here.

In the book you write this:

Our third dd line uses those two labels to do some math: the header length is the value of header_end minus the value of header_start. Because header_start and header_end are just the addresses of places in memory, we can simply subtract to get the distance between those two addresses.

But what are header_end and header_start? I get that they are addresses (and thus, hex numbers), but is header_start going to be 0x00000000 and what is header_end? And what is in between these two addresses, just 0xe85250d6 and 0, or something else?

In the "Paging" section, there is the following paragraph:

"Now, we defined p3_table in the BSS section, which means that it will start as all zeroes. So when we or with 0b11, it means that the first two bits will be set to one, keeping the rest as zeroes."

and the relevant code:

global start

section .text
bits 32
start:
  ; Point the first entry of the level 4 page table to the first entry in the
  ; p3 table
  mov eax, p3_table
  or eax, 0b11
  mov dword [p4_table + 0], eax

section .bss

align 4096

p4_table:
  resb 4096
p3_table:
  resb 4096
p2_table:
  resb 4096

As I understand it, mov eax, p3_table moves the address of p3_table into eax, not its value. So we are not oring 0b11 with zero. If we were, we could just replace all of

mov eax, p3_table
or eax, 0b11
mov dword [p4_table + 0], eax

with the single instruction

mov dword [p4_table + 0], 0b11

Hello, again.

I got as far as linking my boot.asm and multiboot_header.asm scripts. I had errors trying to turn my files into an .iso but I followed the fix here: #35. Now when I try to load my ISO in QEMU I get:

error: no multiboot header found
error: you need to load the kernel first

As far as I can tell, my code is identical to yours in intermezzOS/kernel. You can see my code here: https://github.com/Charlotteis/charlotteOS

Here is the ouput of the os.iso creation, in case that is useful.

charlotte@var-MacBookPro:~/projects/charlotteOS$ grub-mkrescue -o os.iso isofiles
xorriso 1.3.2 : RockRidge filesystem manipulator, libburnia project.

Drive current: -outdev 'stdio:os.iso'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data, 68.4g free
Added to ISO image: directory '/'='/tmp/grub.09DKxz'
xorriso : UPDATE : 548 files added in 1 seconds
Added to ISO image: directory '/'='/home/charlotte/projects/charlotteOS/isofiles'
xorriso : UPDATE : 552 files added in 1 seconds
xorriso : NOTE : Copying to System Area: 512 bytes from file '/usr/lib/grub/i386-pc/boot_hybrid.img'
xorriso : UPDATE :  100.00% done
ISO image produced: 5489 sectors
Written to medium : 5489 sectors at LBA 0
Writing to 'stdio:os.iso' completed successfully.

I am running a fresh install of Ubuntu 14 LTS on a 2012-ish Macbook Pro.

On a Debian system, I was unable to run the ld command from Section 3.2 Hello,World

I was able to resolve this by installing build-essentials. This might need to be added to the Linux setup

I've been reading the book for a few days, and just got stuck at "Setting up a GDT", because the section seems incomplete and contradicting. In short, it is suggesting me to write the following code at "Setting up a code segment" and "Setting up a data segment":

.code: equ $ - gdt64
    dq (1<<44) | (1<<47) | (1<<41) | (1<<43) | (1<<53)
.data: equ $ - gdt64
    dq (1<<44) | (1<<47) | (1<<41)

...yet immediately below, at "Putting it all together", the following code seems to come out of nowhere:

.code: equ $ - gdt64
    dq (1<<44) | (1<<47) | (1<<41)
.data: equ $ - gdt64
    dq (1<<44) | (1<<47) | (1<<41)
    dq (1<<44) | (1<<47) | (1<<41) | (1<<43) | (1<<53)
    dq (1<<44) | (1<<47) | (1<<41)

This should probably be considered as a bug or a TODO.

While most programmers most likely have passing knowledge of hex numbers it might help to include a link to a refresher tutorial or at least mention that the 0x prefix is used for hex numbers.

The book (quite understandably) leans heavily on examples from the Unix world. Including some more Windows friendly examples along with Unix based references may make things more inclusive.

This is to eliminate a warning, but I forget the math.

In section 3.4, "Running in QEMU", when running
$ qemu-system-x86_64 -cdrom os.iso
I get the following when executed over ssh or VM console:
Could not initialize SDL(No available video device) - exiting

Running qemu-system-x86_64 -curses -cdrom os.iso works. Though instructions on how to exit the emulator would be great too.

Debian Jessie on VirtualBox 5.0.0.

By the way, this is a great tutorial. Everything has been explained very well so far.

I just want to start a discussion about if and how help writing intermezzOS could be a thing. I love the project and but I realize this could be a huge time sink for a single person. I have absolutely now experience with assembly for the x86_64 platform, but I am fairly conversant with git and could probably write a large portion of that chapter. I confidently assume that others could contribute in a meaningful way as well.

So my question is. Is collaborative writing something that is interesting, or should it not be pursued. If it is interesting, how do we implement it?

Iv'e read over the paging section a few times, but I still have some conceptual gaps. Can you tell me if I'm understanding things right?

I understand that we use paging to map a 64-bit number to a place in physical memory, which is much smaller than our 64-bit address space.

But you also write in this paragraph:

How should we do our mapping of physical to virtual addresses? You can make this easy, or complex, and it depends on exactly what you want your OS to be good at. Some strategies are better than others, depending on the kinds of programs you expect to be running. We’re going to keep it simple, and use a strategy called ‘identity mapping’. This means that every virtual address will map to a physical address of the same number. Nothing fancy.

This seems to contain a lot of nuance packed in. First does "Some strategies are better than others..." mean there are multiple ways to implement paging? Second, does identity mapping imply that we'll only use as many virtual addresses as we directly have in physical addresses? Meaning that if we have 8GB of ram on computer we'll only have 1,000,000 address spaces to work with in our OS? (Because 8 GB / 64 bit is 1,000,000)? Are there other strategies that let us map 1 physical address to a full range of virtual addresses?

My second questions concern the implementation of paging:

Are we only implementing 4 pages of memory? Since each page is 4096 bytes, that means each page has 512 addresses (4096 bytes / 64 bits), and that's it? Is our OS only working with 2048 addresses? That seems kind of small? But then again if we're only doing this to get our OS into Long Mode, do we just need to set up a single example of paging to trick Grub into running it in 'long mode'. And if the latter is the case, then is our study of paging for a computer memory pertinent to how our OS really handles memory?

http://intermezzos.github.io/book/multiboot-headers.html <- ending tag

In case you get this when trying to boot the ISO image in QEMU:
Could not read from CDROM (code 0009)

on Ubuntu run:

$ sudo apt-get install grub-pc-bin

now try again to make the ISO and run it in QEMU:

$ grub-mkrescue -o os.iso isofiles
$ qemu-system-x86_64 -cdrom os.iso

At the bottom of http://wiki.osdev.org/Bare_Bones is the reason of that error message.

Much of the community seems to prefer rustup to multirust, but the README here still references multirust.

I ran into one particular issue that came from not reloading my shell:

$ nasm -v
NASM version 0.98.40 (Apple Computer, Inc. build 11) compiled on Feb 10 2016

$ brew install nasm
==> Downloading https://homebrew.bintray.com/bottles/nasm-2.11.08.el_capitan.bot
Already downloaded: /Library/Caches/Homebrew/nasm-2.11.08.el_capitan.bottle.tar.gz
==> Pouring nasm-2.11.08.el_capitan.bottle.tar.gz
🍺  /usr/local/Cellar/nasm/2.11.08: 25 files, 2.2M

$ nasm -v
NASM version 0.98.40 (Apple Computer, Inc. build 11) compiled on Feb 10 2016

Now when I go to assemble the multiboot headers in ch 3.1.:

$ nasm -f elf64 multiboot_header.asm
nasm: fatal: unrecognised output format `elf64' - use -hf for a list
type `nasm -h' for help

But if I reload the shell:

$ nasm -v
NASM version 2.11.08 compiled on Sep 22 2015

$ nasm -f elf64 multiboot_header.asm
# assembling is a-ok

I'm not sure why homebrew doesn't put the new nasm in the current context after you install, or if this is just an issue with my laptop specifically. If this is normal behavior, a note to reload the shell at the end of ch 2.2. would be helpful.

In the fourth paragraph there's a sentence that starts with "This book is in the book repository" and it has unexpected line breaks in the middle.

http://intermezzos.github.io/book/hello-world.html

It'd be nice to have a chapter called "troubleshooting" where we collect common problems people had. We could organize it by already-existing-chapter, to make it easier to find a solution to your issue.

Some things that I think the book assumes the reader is 100% comfortable with although it's not unreasonable to think that they might not be:

• different architectures - someone who writes Javascript all day, for example, might be a little unfamiliar with what is meant by the x86 architecture. We should probably include a small section discussing the ideas that there are different CPU architectures, that since we're working at such a low level we have to worry about such things and that we are targeting (at first) just Intel's x86 architecture.
• unix based references - the book (quite understandably) leans heavily on examples from the unix world. Including some more Windows friendly examples along with unix based references may make things more inclusive.
• hexadecimal numbers - while most programmers most likely have passing knowledge of hex numbers it might help to include a link to a refresher tutorial or at least mention that the 0x prefix is used for hex numbers.

Currently a stub. @rylev volunteered to tackle this. Possibly of interest is my partial attempt to define signed integers that didn't make it into another PR:

https://gist.github.com/dariusk/505305e584bd8a2006f5

Hello!

Just doing the 'Hello World' section, allocation 'Hello World' to memory and thus screen.
When counting up the memory spaces, why does it go 8008, 800a, 800c, 800e?

mov word [0xb800a], 0x022c ; ,
mov word [0xb800c], 0x0220 ; 
mov word [0xb800e], 0x0277 ; w

It then moves on to 8010, 8012 etc. What significance do a, c, e have?

"Assembly language gives us direct access to a specific machine. If the basis of computer science is abstraction, the very bottom of the software abstraction layer is assembly. Below it lies only hardware and physics."

This is not strictly true, especially on x86. microcode :(

In the "Automation with Make" chapter, in the "Creating a build subdirectory" section, the reader should also change the line

build: os.iso

in their Makefile to

build: build/os.iso

or make build will not work.

Why do people keep talking about 1 << 5 as if it has the fifth bit set? It's the sixth, and I clarified already.

1 << 0 is 1, with the first bit set; 1 << 1 has bit 1+1=2 set. There are a bunch of errors along these lines in issues, this section, etc. Is this the norm with systems programming?

This is #45 re-opened. A stock arch install with only the basic tools in the book's list installed will give this error. The solution without installing mtools (which is tools for MSDOS disks) is to add a parameter to the grub-mkrescue command.

grub-mkrescue -d /usr/lib/grub/i386-pc/ -o os.iso isofiles

This will tell grub-mkrescue to use BIOS rather than EFI.

http://intermezzos.github.io/book/making-an-iso.html

Here is an excerpt from this section:

Just like before, we are now writing the value in eax to a location. But instead of it being just p2_table + 0, we’re adding ecx * 8 Remember, ecx is our loop counter. Each entry is eight bytes in size: 0b10000011. So we need to multiply the counter by eight, and then add it to p2_table. Let’s take a closer look: let’s assume p2_table is zero, to make the math easier:

p2_table              0    0     0     0     0
ecx                   0    1     2     3     4
ecx * 8               0    8    16    24    32
p2_table + ecx * 8    0    8    16    24    32

We skip eight spaces each time, so we have room for all eight bits of the page table.

"Each entry is eight bytes in size: 0b10000011."

• 0b10000011 is 8 bits, or 1 byte, so I don't see how it's related to each entry being 8 bytes.

"We skip eight spaces each time, so we have room for all eight bits of the page table."

• I think this should say "We skip eight spaces each time, so we have room for all eight bytes of the page table entry."

d73c6b1 introduced a few places where we need some diagrams.

@ag_dubs says she wants to help out here :)

Ashley, let me know if you need help with what the diagrams should look like; i'm just imagining boxes.

Hello, I've been working through the book (it's awesome!) but ran into a problem in section 3.2 when I attempt to invoke ld with the --nmagic flag. Specifically, I see the following output:

$ ld --nmagic --output=kernel.bin --script=linker.ld multiboot_header.o boot.o
ld: unknown option: --nmagic

I took a look at the man page for ld on Mac and couldn't find any documentation for --namgic, although the man page for ld on Linux does document it.

I also read through issue #30 and it seems the flag is essential for continuing. Consequently, I was curious if anyone else has worked through the book on a Mac and ran into a similar issue?

As a side note I also spun up a Linux VM and have been able to run the command successfully on it.

Thanks!

In the paging section of the book, the following bit of text attempts to explain why paging is needed.

Each location in memory has an address, and we can use the address to distinguish between the cells: the value at cell zero, the value at cell ten. But how many cells are there? This question has two answers: the first answer is, how many addresses do we have to hand out? In 64-bit mode, we can create addresses from zero to (2^64) - 1. That’s 18,446,744,073,709,551,616 addresses! We sometimes refer to a sequence of addresses as an ‘address space’, so we might say “The full 64-bit address space has 2^64 addresses.” The other answer is, how much physical RAM do we have in our machine? That will vary per machine. My machine has 8 gigabytes of memory, 8,589,934,592 bytes. But maybe your machine has 4 gigabytes of memory, or sixteen gigabytes of memory. How can we make this work?

It is to me unclear what the problem with this is and why a distinction between virtual and physical addresses will solve it. Perhaps an update of the text is needed to better explain this?

I can't seem to repro locally, but a lot of words seem to be missing spaces between
(e.g. from the first paragraph). I'm on OSX Yosemite with Chrome. Could be it just needs a re-deploy?

in the process of creating the diagrams for what.md, i found a very pleasant pattern that helps aid in learning about how abstractions work and why they are useful. stated generally the pattern is:

• i have A
• A is written explicitly for X
• but i want to support X and Y
• so i put abstraction B in the middle

knowing this, creating the diagram for hypervisors posed a challenge. it doesn't entirely follow this pattern. given that this is an intro chapter, i would recommend:

• removing hypervisors from this chapter
• writing a conclusing paragraph that states this pattern to reinforce what was presented in the chapter

given this, we should close #2 as the hypervisor diagram isn't need right now

Don't know if you're aware, don't know if it was intentional, but the first Makefile had a build target of build: os.iso that disappears in the translation to build directories in the final Makefile. Just... heads up.

We should be pinning to a version so we know that it works.

STR:

1. Visit http://intermezzos.github.io/book/
2. View source (Ctrl+U in Firefox)

ACTUAL RESULTS:
Page source is all on one line, so it's not easily viewable/scrollable.

EXPECTED RESULTS:
Somewhat-easier-to-view page source, with periodic linebreaks, e.g. maybe at the boundaries between major pieces of the page at least.

I presume this page is autogenerated from markdown. I'm hoping/guessing that whatever markdown parser you're using will have that as a feature.

At least, the native GitHub markdown view does insert newlines into the HTML page source, as you can see if you view the source of https://github.com/intermezzOS/book/blob/master/src/preface.md and scroll down. This is only an issue on the actual intermezzos.github.io-backed version of the page.

(Note that this is distinct from #1, though both issues are about linebreaks.)

http://intermezzos.github.io/book/multiboot-headers.html

Thanks for the tutorial! One note, though: grub-mkrescue is not installed with certain versions of xorriso (notably the version in the Ubuntu 14.04 package repository). It might be worth mentioning that if you don't have grub-mkrescue after installing everything else, you should install grub-common (at least that's the appropriate Ubuntu package...).